Medical service system, device, equipment and method

Abstract

The embodiment of the invention provides a medical service system, device, equipment and method, the system comprises processing equipment, sensing equipment and medical service equipment, and the sensing equipment and the medical service equipment are in communication connection with the processing equipment and are applied to a target medical scene; the sensing device is configured to collect sensing information related to a target medical scene and send the sensing information to the processing device; and the processing equipment is configured to maintain an intelligent agent corresponding to the medical service equipment and realize user service related to the target medical scene by enabling the intelligent agent to participate in processing the perception information.

Description

Description of the case

[0001] This application is a divisional application for the Chinese application with application date of December 31, 2024, application number 202411993721.7, and invention name “A medical service system, device, equipment and method”. The Chinese application claims the priority of international applications with application numbers PCT / CN2024 / 109057, PCT / CN2024 / 109058, PCT / CN2024 / 109059 and PCT / CN2024 / 109065 filed on July 31, 2024, all of which are incorporated herein by reference. Technical Field

[0002] This specification relates to the field of medical services, and in particular to a medical service system, device, equipment and method built based on XR technology, AI technology and / or other pioneering technologies. Background Art

[0003] Currently, when a patient needs to receive multiple medical services in a hospital, he / she needs to go through a complex and fragmented medical service process. This is cumbersome and time-consuming for patients, making it more complicated for patients to understand the medical service process and identify appropriate facilities. As a result, the overall efficiency of medical services is reduced. At the same time, these complexities also bring major challenges to hospital personnel in providing medical services and managing related medical resources.

[0004] The rapid development of cutting-edge technologies, such as extended reality (XR) technology (e.g., augmented reality (AR) technology, virtual reality (VR) technology, mixed reality (MR) technology, etc.), artificial intelligence (AI) technology, digital twin technology, Internet of Things (IoT) technology, blockchain technology, spatial computing technology, image rendering technology, etc., have triggered transformative changes in different systems and brought substantial benefits. However, despite these advances, the full utilization of these innovative technologies in hospital systems has not yet been achieved. Therefore, it is necessary to develop medical service systems, devices, equipment, and methods that integrate these advanced technologies to improve service efficiency, service accuracy, and patient satisfaction (e.g., patient experience, especially immersive medical service experience). Summary of the invention

[0005] One aspect of an embodiment of the present specification provides a medical service system. The system includes a processing device, a perception device and a medical service device, wherein: the perception device and the medical service device are respectively connected to the processing device in communication and applied to a target medical scenario; the perception device is configured to collect perception information related to the target medical scenario and send the perception information to the processing device; and the processing device is configured to maintain a first agent corresponding to the medical service device, and implement user services related to the target medical scenario by enabling the first agent to participate in processing the perception information. The first agent achieves self-evolution based on historical service data related to the target medical scenario and artificial intelligence technology. In order to implement user services related to the target medical scenario, the processing device is configured to obtain patient data related to patients in the target medical scenario, and implement the user services by enabling the first agent to participate in processing the perception information and patient data related to the patient.

[0006] One aspect of an embodiment of the present specification provides a medical service system. The system includes a processing device, a sensing device, and a clinic terminal device. The sensing device and the clinic terminal device are respectively connected to the processing device in communication, and are applied to a target medical scenario, wherein the target medical scenario involves a medical consultation service provided to a patient in a clinic. The sensing device is configured to collect sensing information related to the target medical scenario and send the sensing information to the processing device. The processing device is configured to maintain a consultation agent corresponding to the clinic terminal device, and implement user services related to the target medical scenario by enabling the consultation agent to participate in processing the sensing information. The consultation agent realizes self-evolution based on historical service data related to the target medical scenario and artificial intelligence technology. The user service includes at least one of the medical data display, providing medical advice, generating diagnostic records, and remote accompanying services.

[0007] One aspect of an embodiment of the present specification provides a medical service system. The system includes a processing device, a perception device, and a pre-consultation terminal device. The perception device and the pre-consultation terminal device are communicatively connected to the processing device and applied to a target medical scenario, which involves a pre-consultation service provided to a patient. The perception device is configured to collect perception information related to the target medical scenario and send the perception information to the processing device. The processing device is configured to maintain a pre-consultation intelligent agent corresponding to the pre-consultation terminal device, and implement the pre-consultation service by enabling the pre-consultation intelligent agent to participate in processing the perception information. To implement the pre-consultation service, the pre-consultation intelligent agent is configured to: based on the department where the doctor registered by the patient is located, display a virtual character through the pre-consultation terminal device to conduct a pre-consultation inquiry on the patient; and generate a pre-consultation record based on the perception information collected by the perception device in the pre-consultation inquiry.

[0008] One aspect of an embodiment of the present specification provides a medical service system. The system includes a processing device, a sensing device and a hospital bed. The hospital bed includes a bed body and a terminal device provided to the patient. The sensing device and the terminal device are connected to the processing device in communication and applied to a target medical scenario, which involves hospitalization services provided to patients in a ward. The sensing device is configured to collect sensing information related to the target medical scenario and send the sensing information to the processing device. The processing device is configured to maintain an inpatient agent corresponding to the hospital bed and implement the inpatient service by enabling the inpatient agent to participate in processing the sensing information. The inpatient agent realizes self-evolution based on historical service data related to the target medical scenario and artificial intelligence technology. To implement the inpatient service, the inpatient agent is configured to: monitor a data source for collecting data related to the patient's hospitalization process, the data source at least including the sensing device; in response to detecting a data update in at least one of the data sources, perform one or more preset operations corresponding to an event of interest based on the updated data obtained from the at least one data source, the updated data at least including the sensing information.

[0009] One aspect of an embodiment of the present specification provides a medical service system. The system includes a processing device, a perception device, and an intelligent nursing cart with an automatic driving function. The perception device and the intelligent nursing cart are communicatively connected to the processing device and applied to a target medical scenario, wherein the target use scenario involves nursing services provided to patients in a ward. The perception device is configured to collect perception information related to the target medical scenario and send the perception information to the processing device. The processing device is configured to maintain a nursing agent corresponding to the intelligent nursing cart, and implement the nursing service by enabling the nursing agent to participate in processing the perception information. The nursing agent achieves self-evolution based on historical service data related to the target medical scenario and artificial intelligence technology. To implement the nursing service, the nursing agent is configured to: determine whether the patient meets the conditions for admission examination after staying in the ward based on the perception information; in response to determining that the patient meets the conditions for admission examination, control the smart nursing cart to guide the nurse to the ward to perform the admission examination for the patient; for each day that the patient is hospitalized, determine the patient's daily plan, wherein the daily plan includes at least one nursing operation that needs to be performed on the patient on that day; and for each of the at least one nursing operation, control the smart nursing cart to guide the nurse to the ward to perform the nursing operation based on the planned time of the nursing operation.

[0010] One aspect of an embodiment of the present specification provides a medical service system. The system includes a processing device, a sensing device, and an intelligent surgical terminal device. The sensing device and the intelligent surgical terminal device are communicatively connected to the processing device and applied to a target medical scenario, which involves surgical services provided to patients in an operating room. The sensing device is configured to collect sensing information related to the target medical scenario and send the sensing information to the processing device. The processing device is configured to maintain a surgical agent corresponding to the intelligent surgical terminal device, and implement the surgical service by enabling the surgical agent to participate in processing the sensing information. The surgical agent realizes self-evolution based on historical service data related to the target medical scenario and artificial intelligence technology. To implement the surgical service, the surgical agent is configured to: obtain the sensing information collected by the sensing device during the patient's surgery; and perform preset operations corresponding to events of interest.

[0011] One aspect of the embodiments of this specification provides a medical service system. The system includes a processing device, a sensing device and an intelligent mechanical nurse. The sensing device and the intelligent mechanical nurse are connected to the processing device in communication and applied to a target medical scenario, which involves a surgical service provided to a patient in an operating room. The sensing device is configured to collect sensing information related to the target medical scenario and send the sensing information to the processing device. The processing device is configured to maintain a mechanical nurse agent corresponding to the intelligent mechanical nurse, and implement the surgical service by enabling the mechanical nurse agent to participate in processing the sensing information. The mechanical nurse agent realizes self-evolution based on historical service data related to the target medical scenario and artificial intelligence technology. To implement the surgical service, the mechanical nurse agent is configured to: obtain the surgical plan of the patient; based on the surgical plan, control the intelligent mechanical nurse to prepare surgical tools in the operating room before the operation; or based on the sensing information, control the intelligent mechanical nurse to assist the operation; or after the operation, control the intelligent mechanical nurse to organize the operating room.

[0012] Additional features will be described in the following description, and some of the additional characteristics of this specification will be apparent to those skilled in the art through study of the following description and corresponding drawings or understanding of the production or operation of the embodiments. The features of this specification can be realized and achieved through the practice or use of the methods, means and combinations of various aspects of the specific embodiments described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present specification is further described according to exemplary embodiments. These exemplary embodiments are described in detail with reference to the accompanying drawings. These embodiments are non-limiting exemplary embodiments, in which the same reference numerals represent similar structures throughout the multiple views of the drawings, and in which:

[0014] Figure 1 is a block diagram of an exemplary medical service system according to some embodiments of the present specification;

[0015] Figure 2 is a schematic diagram of an exemplary medical service system according to some embodiments of the present specification;

[0016] Figure 3 is a schematic diagram of an exemplary hospital support platform according to some embodiments of the present specification;

[0017] Figure 4 is a block diagram of an exemplary processing device according to some embodiments of the present specification;

[0018] Figure 5is a schematic diagram of an exemplary process of providing user services according to some embodiments of this specification;

[0019] Figure 6 is a schematic diagram of an exemplary medical consultation process according to some embodiments of this specification;

[0020] Figure 7 is an exemplary schematic diagram of determining a doctor for patient registration according to some embodiments of the present specification;

[0021] Figure 8 is a schematic diagram of an exemplary process for providing pre-consultation services according to some embodiments of this specification;

[0022] Fig. 9 is a schematic diagram of an exemplary process of performing a second inquiry according to some embodiments of this specification;

[0023] Fig.10 is a schematic diagram of an exemplary process for providing medical consultation services based on perception information according to some embodiments of this specification;

[0024] Fig.11 is a schematic diagram of an exemplary medical consultation interface according to some embodiments of this specification;

[0025] Fig.12 is a schematic diagram of an exemplary hospitalization process according to some embodiments of this specification;

[0026] Fig.13 is an exemplary flow chart of providing inpatient services according to some embodiments of the present specification;

[0027] Fig.14 is a schematic diagram of an exemplary process for providing inpatient services according to some embodiments of the present specification;

[0028] Fig.15 is a schematic diagram of an exemplary process of providing admission inquiry services according to some embodiments of this specification;

[0029] Fig.16 is a schematic diagram of an exemplary process for providing nursing services according to some embodiments of the present specification;

[0030] Fig.17 is a schematic diagram of an exemplary process of providing ward rounds service according to some embodiments of this specification;

[0031] Fig.18 is an exemplary schematic diagram of surgical planning and execution according to some embodiments of the present specification;

[0032] Fig.19 is a schematic diagram of an exemplary process of performing a surgery according to some embodiments of the present specification;

[0033] Fig. 20 It is a schematic diagram of an exemplary process of preoperative preparation and surgical execution according to other embodiments of the present specification. Specific implementation methods

[0034] In the detailed description that follows, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that the present specification may be implemented without these details. In other cases, in order to avoid unnecessarily obscuring aspects of the present specification, well-known methods, processes, systems, components and / or circuits have been described at a higher level. Various modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined in the present specification may be applied to other embodiments and applications without departing from the spirit and scope of the present specification. Therefore, the present specification is not limited to the embodiments shown, but is in accordance with the widest scope consistent with the claims.

[0035] The terms used in this specification are only used to describe specific example embodiments and are not intended to be limiting. As used in this specification, the singular forms "a", "an", and "the" may also be intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "include" and / or "comprise" when used in this specification specify the presence of stated features. Integers, steps, operations, elements and / or components, but do not exclude the presence and addition of at least one other feature, integer, step, operation, element, component and / or combination thereof.

[0036] It should be understood that when a unit, engine, module or block is referred to as being "on," "connected to," or "coupled to" another unit, engine, module or block, it may be directly connected or coupled to, or in communication with, the other unit, engine, module or block, or there may be intermediate units, engines, modules or blocks, unless the context clearly indicates otherwise. As used in this specification, the term "and / or" includes any and all combinations of at least one of the associated listed items.

[0037] The features and characteristics of this specification, as well as the operation and function of the related structural elements and the combination and manufacturing economy of the parts, will become more apparent after consideration with reference to the following description. All drawings constitute a part of this specification. However, it should be expressly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should be understood that the drawings are not drawn to scale.

[0038] Figure 1is a block diagram of an exemplary medical service system 100 according to some embodiments of the present specification.

[0039] The medical service system 100 can also be called a meta-hospital system. It is built based on a variety of innovative technologies, including metaverse technology, XR technology (for example, AR technology, VR technology, MR technology, etc.), AI technology, digital twin technology, IoT technology, data circulation technology (for example, blockchain technology, data privacy computing technology), spatial computing technology, image rendering technology, etc.

[0040] like Figure 1 As shown, the medical service system 100 may include a physical hospital 110, a virtual hospital 130, a user space application 120, and a hospital support platform 140. In some embodiments, the hospital support platform 140 may map data related to the physical hospital 110 to the virtual hospital 130 corresponding to the physical hospital 110, and provide user services to related users of the physical hospital 110 through the user space application 120. For example, at least a portion of the user services may be provided to the related users based on the interaction between the related users and the virtual hospital 130.

[0041] Physical hospital 110 refers to a hospital that exists in the physical world and has tangible properties (e.g., has measurable properties such as mass, volume, shape, and can be sensed by human senses or instruments). In this article, health care institutions that provide medical, surgical, and psychiatric care and treatment to people are collectively referred to as hospitals.

[0042] like Figure 1 As shown, the physical hospital 110 may include multiple physical entities. For example, the multiple physical entities may include departments, users, hardware equipment, user services, public areas, medical service processes, etc., or any combination thereof.

[0043] A department refers to a specialized unit or department that specializes in providing a specific type of medical care, treatment, and services. Each department may focus on a specific medical field and may be staffed with healthcare professionals with expertise in that field. For example, according to the medical service process corresponding to each department, the department may include an outpatient department, an inpatient department, a surgical department, a support department (e.g., a registration department, a pharmacy department), etc., or any combination thereof. As another example, according to the medical field corresponding to each department, the department may include internal medicine, surgery, specialist medicine, child health care, etc., or any combination thereof.

[0044] The user may include any user associated with the physical hospital 110 (or referred to as a related user of the physical hospital 110). For example, the user may include a patient (or a part of a patient (e.g., an organ)), a companion, a visitor of a patient, a hospital staff of the physical hospital 110, a supplier of the physical hospital 110, an application developer of the physical hospital 110, etc., or any combination thereof. The hospital staff of the physical hospital 110 may include medical service providers (e.g., doctors, nurses, technicians, etc.), hospital administrators, support staff, or the like, or any combination thereof. Exemplary hospital administrators may include department nursing administrators, clinical administrators, department directors, hospital directors, hospital administrators, functional administrators, or the like, or any combination thereof.

[0045] The hardware devices may include hardware devices located in the physical hospital 110 and / or hardware devices that communicate with hardware devices in the physical hospital 110. Exemplary hardware devices may include terminal devices, medical service devices, sensing devices, basic devices, etc., or any combination thereof.

[0046] The terminal device may include a terminal device that interacts with a user of the medical service system 100. For example, the terminal device may include a terminal device that interacts with a patient (also referred to as a patient terminal device), a terminal device that interacts with a patient's doctor (also referred to as a doctor terminal device or a second terminal device), a terminal device device that interacts with a nurse (also referred to as a nurse terminal device), a terminal device device that interacts with a remote visitor (also referred to as a remote terminal device), or a public terminal device of a hospital (e.g., a consulting room terminal device, a bedside terminal device, a terminal device in a waiting area, an intelligent surgical terminal device), etc., or any combination thereof. In this specification, unless clearly obtained from the context or otherwise specified by the context, terminal devices owned by a user and terminal devices provided to a user by the physical hospital 110 are collectively referred to as terminal devices of the user or terminal devices that interact with the user.

[0047] The terminal device may include a mobile terminal device, an XR device, a smart wearable device, etc. The mobile terminal device may include a smart phone, a personal digital assistant (PDA), a display, a gaming device, a navigation device, a handheld terminal device (POS), a tablet computer, etc., or any combination thereof.

[0048] XR devices may include devices that allow users to participate in extended reality experiences. For example, XR devices may include VR components, AR components, MR components, etc., or any combination thereof. In some embodiments, XR devices may include XR helmets, XR glasses, XR patches, stereo headphones, etc., or any combination thereof. For example, XR devices may include Google Glass TM 、Oculus Rift TM 、GearVRTM 、AppleVisionpro TM Etc. Specifically, the XR device may include a display component on which virtual content can be presented and / or displayed. In some embodiments, the XR device may further include an input component. The input component enables user interaction between the user and the virtual content (e.g., a virtual surgical environment) displayed by the display component. For example, the input component may include a touch sensor, a microphone, an image sensor, etc., which is configured to receive user input, which can be provided to the XR device and used to control the virtual world by changing the visual content presented on the display component. The input component may include a handle, a glove, a stylus, a console, etc.

[0049] Smart wearable devices may include smart bracelets, smart shoes and socks, smart glasses, smart helmets, smart watches, smart clothes, smart backpacks, smart accessories, etc., or any combination thereof. In some embodiments, smart wearable devices may obtain physiological data (e.g., heart rate, blood pressure, body temperature, etc.) of the user.

[0050] The medical service equipment can be configured to provide medical services to patients. For example, the medical service equipment can include examination equipment, nursing equipment, treatment equipment, etc., or any combination thereof.

[0051] The inspection device may be configured to provide inspection services to the patient, for example, to collect the patient's inspection data. Exemplary inspection data may include heart rate, respiratory rate, body temperature, blood pressure, medical imaging data, body fluid test reports (e.g., blood test reports), etc., or any combination thereof. Accordingly, the inspection device may include a vital sign monitor (e.g., a blood pressure monitor, a blood glucose meter, a heart rate meter, a thermometer, a digital stethoscope, etc.), a medical imaging device (e.g., a computer tomography (CT) device, a digital subtraction angiography (DSA) device, a magnetic resonance (MR) device, etc.), a laboratory device (e.g., a routine blood test device, etc.), or any combination thereof.

[0052] Nursing equipment can be configured to provide nursing services to patients and / or assist medical service providers in providing nursing services. Exemplary nursing equipment can include hospital beds, patient care robots, smart nursing carts, smart medicine boxes, smart wheelchairs, etc.

[0053] The therapeutic device can be configured to provide therapeutic services to patients and / or assist medical service providers in providing therapeutic services. Exemplary therapeutic devices can include surgical devices, radiation therapy devices, physical therapy devices, etc., or any combination thereof.

[0054] The perception device may be configured to collect perception information related to its environment (e.g., target medical scene). For example, the perception device may include an image sensor, a sound sensor, etc. The image sensor may be configured to collect image data in the physical hospital 110, and the sound sensor may be configured to collect voice signals in the physical hospital 110. In some embodiments, the perception device may be an independent device or integrated into another device. For example, the sound sensor may be part of a medical service device.

[0055] The infrastructure can be configured to support data transmission, storage and processing. For example, the infrastructure can include networks, computer room facilities, computing equipment, computing chips, storage devices, etc.

[0056] In some embodiments, at least a portion of the hardware devices of the physical hospital 110 are Internet of Things (IoT) devices. IoT devices refer to devices with sensors, processing capabilities, software, and other technologies that connect and exchange data with other devices and systems via the Internet or other communication networks. For example, one or more medical service devices and / or sensing devices of the physical hospital 110 are IoT devices and are configured to transmit collected data to the hospital support platform 140 for storage and / or processing.

[0057] User services may include any services provided to users by the hospital support platform 140. For example, user services include medical services provided to patients and / or accompanying persons, support services provided to staff of the physical hospital 110 and / or suppliers of the physical hospital 110, etc. In some embodiments, user services may be provided to patients, doctors, and hospital administrators through the user space application 120. For more information about this part, please refer to the relevant description below.

[0058] Public areas refer to shared spaces accessible to users (or portions of users) in the physical hospital 110. For example, public areas may include a reception area (eg, front desk), a waiting area, a corridor, or any combination thereof.

[0059] The medical service process refers to the process of providing corresponding medical services to patients. The medical service process usually includes several links and / or steps, and the user needs to go through these links and / or steps to obtain the corresponding medical services. Exemplary medical service processes may include consultation processes, hospitalization processes, surgical processes, or similar processes, or any combination thereof. In some embodiments, the medical service process may include medical service processes corresponding to different departments, different diseases, etc. In some embodiments, a preset data collection protocol can be set and the standard links involved in the medical service process and how to collect data related to the medical service process can be specified. For more information about the medical service process, please refer to this manual. Figure 5 and its related description.

[0060] The user space application 120 provides users with access to user services provided by the hospital support platform 140. The user space application 120 can be an application, a plug-in, a website, a small program, or any other suitable form. For example, the user space application 120 is an application installed on a user terminal device, and the application includes a user interface for the user to initiate a request and receive the corresponding service.

[0061] In some embodiments, the user space application 120 may include different applications corresponding to different types of users. For example, the user space application 120 includes a patient space application corresponding to the patient, a medical space application corresponding to the doctor, a management space application corresponding to the manager, etc., or any combination thereof. The user services provided by the patient space application, the medical space application, and the management space application are also referred to as patient space services, medical space services, and management space services, respectively. Exemplary patient space services include registration services, path guidance services, pre-consultation services, remote outpatient services, hospitalization services, discharge services, etc. Exemplary medical space services include scheduling services, surgical planning services, surgical simulation services, patient management services, remote ward rounds services, remote outpatient services, etc. Exemplary manager space services include monitoring services, medical service evaluation services, equipment parameter setting services, service parameter setting services, resource scheduling services, etc.

[0062] In some embodiments, the patient space application, the medical space application, and the management space application may be integrated into one user space application 120, and the user space application 120 may be configured to provide access portals for each type of user (e.g., patients, medical service providers, managers, etc.). As an example only, a specific user may have a corresponding account that can be used to log in to the user space application, view corresponding diagnosis and treatment data, and obtain corresponding user services.

[0063] In some embodiments of the present specification, by providing user space applications for different types of users, each type of user can easily obtain various user services that he / she may need on his / her corresponding user space application. In addition, currently users usually need to install various applications to obtain different user services, which leads to poor user experience and high development costs. Therefore, the user space application of the present specification can improve user experience, improve service quality and efficiency, enhance service security, and reduce development or operation costs.

[0064] In some embodiments, the user space application 120 can be configured to provide relevant users of the physical hospital 110 with access to interact with the virtual hospital 130. For example, through the user space application 120, the user can enter instructions for retrieving the digital content of the virtual hospital 130 (e.g., hardware equipment, patient organs, digital twin models of public areas), view the digital content, and interact with the digital content. As another example, through the user space application 120, the user can communicate with a virtual character representing an intelligent body. In some embodiments, the public terminal device of the hospital can install a tube space application, and the administrator account of the department corresponding to the public terminal device can be logged in in the tube space application. Users can receive user services through the tube space application installed in the public terminal device.

[0065] The virtual hospital 130 is a digital twin (ie, a virtual representation or virtual copy) of the physical hospital 110 and is used to simulate, analyze, predict, and optimize the operating status of the physical hospital 110. For example, the virtual hospital 130 may be a real-time digital copy of the physical hospital 110.

[0066] In some embodiments, the virtual hospital 130 may be presented to the user using digital technology. For example, when the relevant user interacts with the virtual hospital 130, at least a portion of the virtual hospital 130 may be presented to the relevant user using XR technology. As an example only, at least a portion of the virtual hospital 130 may be superimposed on the relevant user's real-world view using MR technology.

[0067] In some embodiments, the virtual hospital 130 may include a digital twin of a physical entity associated with the physical hospital 110. A digital twin refers to a virtual representation of a physical entity (e.g., a virtual copy, a mapping, a digital simulator). A digital twin can reflect and predict the state, behavior, and performance of a physical entity in real time. For example, the virtual hospital 130 may include a digital twin of at least a portion of the medical services, departments, users, hardware equipment, user services, public areas, medical service processes, etc. of the physical hospital 110. The digital twin of a physical entity may be in a variety of forms, including models, images, graphics, text, numerical values, and the like. For example, a digital twin may be a virtual hospital corresponding to a physical hospital, virtual personnel (e.g., virtual doctors, virtual nurses, and virtual patients) corresponding to personnel entities (e.g., doctors, nurses, and patients), virtual equipment (e.g., virtual imaging equipment and virtual scalpels) corresponding to medical service equipment (e.g., imaging equipment and scalpels), and the like.

[0068] In some embodiments, the digital twin may include one or more first digital twins and / or one or more second digital twins. The state of each first digital twin may be updated based on the update of the state of the corresponding physical entity. For example, in the process of mapping data related to the physical hospital 110 to the virtual hospital 130, one or more first digital twins may be updated. One or more second digital twins may be updated by at least one of the user space applications 120, and the update of each second digital twin may result in an update of the state of the corresponding physical entity. When the corresponding physical entity changes its state, the first digital twin may be updated accordingly; when the second digital twin is updated, the state of the corresponding physical entity also changes accordingly. For example, one or more first digital twins may include digital twins of public areas, medical services, users, hardware devices, etc., and one or more second digital twins may include digital twins of hardware devices, user services, medical service processes, etc. It should be understood that the digital twin may be a first digital twin or a second digital twin.

[0069] In some embodiments of the present specification, by generating a virtual hospital 130 that includes a digital twin of physical entities related to the physical hospital 110, the physical hospital 110 (including hardware equipment, users, user services, medical service processes, etc.) can be simulated and tested in a safe and controllable environment. Through virtual reality linkage (for example, real-time interaction between the physical hospital 110 and the virtual hospital 130), various medical scenarios can be predicted and responded to more accurately, thereby improving the quality and efficiency of medical services. In addition, the use of XR technology and virtual reality integration technology makes the interaction of relevant users more natural and intuitive, providing a more comfortable and efficient medical environment, thereby enhancing the user experience.

[0070] In some embodiments, the virtual hospital 130 may further include an intelligent agent that self-evolves based on data related to the physical hospital 110 and AI technology.

[0071] An agent refers to an agent that acts in an intelligent manner. For example, an agent may include a computing / software entity that can autonomously learn and evolve, and perceive and analyze data to perform specific tasks and / or achieve specific goals (e.g., medical service processes). Through AI technology (e.g., reinforcement learning, deep learning, etc.), the agent can continuously learn and self-optimize in interaction with the environment. In addition, the agent can collect and analyze massive data (e.g., relevant data of the physical hospital 110) through big data technology, and mine patterns and learning rules from the data to optimize the decision-making process, thereby identifying environmental changes in uncertain or dynamic environments, responding quickly, and making reasonable judgments. For example, the agent can autonomously learn and evolve based on AI technology to adapt to changes in the physical hospital 110. As an example only, the agent can be built based on NLP technology (e.g., large language models, etc.), and can automatically learn and autonomously update through a large amount of language text (e.g., hospital business data and patient feedback information) to improve the quality of user services provided by the physical hospital 110.

[0072] In some embodiments, agents may include different types of agents corresponding to different medical service processes, different user services, different departments, different diseases, different hospital positions (e.g., nurses, doctors, technicians, etc.), different links in the medical service process, etc. Agents of a specific type are used to process tasks corresponding to a specific type. In some embodiments, an agent may correspond to different medical service processes (or different medical services, or different departments, or different diseases, or different hospital locations). In some embodiments, agents may operate with reference to basic configuration data (e.g., dictionaries, knowledge graphs, templates, etc.) of the departments and / or diseases corresponding to the agents. In some embodiments, multiple agents may collaborate and share information through network communications to jointly complete complex tasks.

[0073] In some embodiments, the agent includes a first agent and / or a second agent. Each first agent corresponds to a medical service device and is configured to provide user services related to the target medical scene where the medical service device is located. For example, the first agent includes an agent corresponding to a smart bed, which can provide inpatient services in the ward by processing the sensory information collected by the sensory device in the ward. In some embodiments, the first agent realizes self-evolution based on historical service data related to the target medical scene and artificial intelligence technology.

[0074] Each second agent corresponds to a sensory device. In some embodiments, the second agent can be used to control the sensory device to collect sensory information. For example, the second agent can learn the best sensory information collection mechanism (e.g., what sensory information needs to be collected, how to collect sensory information, quality standards of sensory information, etc.) from historical service data, and control the sensory device based on the learned mechanism.

[0075] In some embodiments, the configuration of the agent can be set. For example, basic configuration data used by the agent in operation can be set. The basic configuration data can include dictionaries, knowledge databases, templates, etc. For another example, the use rights of the agent can be set for different users. In some embodiments, the administrator of the physical hospital 110 can set the configuration of the agent through the management space application.

[0076] In some embodiments, the agent can be integrated into or deployed on a hardware device. For example, the agent corresponding to the inpatient service can be integrated into a hospital bed or a presentation device of a hospital bed. In some embodiments, the agent can be integrated into or deployed on an embodied intelligent robot. An embodied intelligent robot refers to a robot system that combines physical existence (embodiment) with intelligent behavior (cognition). The embodied intelligent robot can be configured to interact with the real world in a way that imitates or supplements human capabilities, using physical form and cognitive functions to perform tasks, make decisions, and adapt to the environment. By utilizing artificial intelligence and sensor technology, the embodied intelligent robot can operate autonomously, interact with the environment, and continuously improve performance. For example, the embodied intelligent robot can be configured as an agent corresponding to a surgical service and assist a doctor in performing surgery. For another example, a sensing device and a processing device (e.g., processing device 210) can be integrated into an embodied intelligent robot, and at least a portion of the user service can be provided to relevant users of the target medical scene by the embodied robot.

[0077] In some embodiments, at least a portion of user services may be provided based on an agent. For example, at least a portion of user services may be provided to relevant users based on a processing result, wherein the processing result is generated by at least one of the agents based on data related to the physical hospital 110. As an example only, the data related to the physical hospital 110 may include data related to a medical service process of the physical hospital 110, the agent may include an agent corresponding to the medical service process, and user services may be provided to relevant users of the medical service process by processing the data using the agent corresponding to the medical service process.

[0078] The hospital support platform 140 may be configured to provide technical support to the medical service system 100. For example, the hospital support platform 140 may include computing hardware and software to support innovative technologies, including XR technology, AI technology, digital twin technology, data circulation technology, etc. In some embodiments, the hospital support platform 140 may include at least a storage device for data storage and a processing device for data calculation.

[0079] In some embodiments, the hospital support platform 140 can support the interaction between the physical hospital 110 and the virtual hospital 130. For example, the processing device of the hospital support platform 140 can obtain data related to the physical hospital 110 from the hardware device, and map the data related to the physical hospital 110 to the virtual hospital 130. For example, the processing device of the hospital support platform 140 can update a part of the digital twin in the virtual hospital 130 (e.g., one or more first digital twins) based on the obtained data, so that each part of the digital twin in the virtual hospital 130 can reflect the updated state of the corresponding physical entity in the physical hospital 110. Based on this digital twin that is constantly updated with the corresponding physical entity, the user can understand the state of the physical entity related to the physical hospital 110 in real time, thereby realizing the monitoring and evaluation of the physical entity. As an example only, the processing device of the hospital support platform 140 can monitor the real-time operation data of the medical service equipment, and update the digital twin corresponding to the medical service equipment based on the real-time operation data. As another example, an agent corresponding to data related to the physical hospital 110 can be trained and / or updated based on the data related to the physical hospital 110, thereby self-evolving and self-learning.

[0080] In some embodiments, the hospital support platform 140 may support and / or provide user services to relevant users of the physical hospital 110. For example, in response to receiving a user service request from a user, a processing device of the hospital support platform 140 may provide a user service corresponding to the service request. For another example, in response to detecting that a user service needs to be provided to a user, the processing device of the hospital support platform 140 may control a physical entity or a virtual entity corresponding to the user service to provide the user service. For example, in response to detecting that an abnormality exists in the physiological condition of a patient, the processing device of the hospital support platform 140 may control a smart nursing cart to guide a nurse to the patient's ward.

[0081] In some embodiments, at least a portion of user services may be provided to the relevant user based on the interaction between the relevant user and the virtual hospital 130. Interaction refers to the interaction or influence (e.g., conversation, behavior, etc.) between the relevant user and the virtual hospital 130. For example, the interaction between the relevant user and the virtual hospital 130 may include the interaction between the relevant user and the digital twin in the virtual hospital 130, the interaction between the relevant user and the intelligent agent, the interaction between the relevant user and the virtual character, etc., or any combination thereof.

[0082] In some embodiments, at least a portion of user services may be provided to the relevant user based on the interaction between the relevant user and at least one of the digital twins. For example, an update instruction for a second digital twin input by a relevant user may be received through the user space application 120, and the corresponding physical entity of the second digital twin may be updated according to the update instruction. As another example, a user may view a first digital twin of a physical entity (e.g., a 3D digital twin model of a patient's organ or a hardware device) through the user space application 120 to understand the state of the physical entity. For example, the user may change the display angle, display size, etc. of the digital twin.

[0083] In some embodiments, the processing device of the hospital support platform 140 can present a virtual character corresponding to the intelligent agent through a user space application, interact with the relevant user, and provide at least part of the user service to the relevant user based on the interaction between the relevant user and the virtual character.

[0084] In some embodiments, the hospital support platform 140 may have a five-layer structure, including a hardware device layer, an interface layer, a data processing layer, an application development layer, and a service layer. In some embodiments, the hardware device of the physical hospital 110 may be part of the hospital support platform 140. For more information about the hospital support platform, please refer to the specification. Figure 3 and its related description.

[0085] In some embodiments of the present specification, a virtual hospital corresponding to the physical hospital can be established by comprehensively integrating various internal and external resources of the physical hospital (e.g., medical service equipment, hospital personnel, medicines and consumables, etc.). The virtual hospital can reflect the real-time status (e.g., changes, updates, etc.) of the physical entities related to the physical hospital, thereby enabling monitoring and evaluation of the physical entities. This integration can provide accurate data support for the operation and intelligent decision-making of medical services. In addition, through the virtual hospital, users related to medical services can jointly establish an open and shared ecosystem, thereby promoting the innovation and improvement of medical services.

[0086] In addition, it is possible to provide full-life cycle healthcare services for patients that are linked inside and outside the hospital. The perspective of medical services has expanded from simple disease treatment to cover the entire life cycle of patients, including prevention, diagnosis, treatment, rehabilitation, health management, etc. By establishing linkages inside and outside the hospital, physical hospitals can better integrate online and offline resources to provide patients with comprehensive and continuous medical and health services. For example, through remote monitoring and online consultations, patients' health conditions can be followed up in real time, treatment plans can be adjusted in a timely manner, and treatment effects can be improved.

[0087] Figure 2 is a schematic diagram of an exemplary medical service system 200 according to some embodiments of the present specification.

[0088] like Figure 2 As shown, the medical service system 200 may include a processing device 210, a network 220, a storage device 230, one or more medical service devices 240, one or more sensing devices 250, one or more patient terminal devices 260 of a patient 261, and one or more doctor terminal devices 270 of a doctor 271 associated with the patient 261. In some embodiments, the components in the medical service system 200 may be connected and / or communicated with each other via wireless connection, wired connection, or a combination thereof. For example, one or more sensing devices 250 and one or more medical service devices 240 may be connected to the processing device 210 in communication and used in a target medical scenario. The target medical scenario refers to a scenario where at least one medical service is provided. For example, the target medical scenario may occur in a hospital or a part of a hospital, such as a consulting room, a ward, a waiting room, an operating room, etc. The connection between the components of the medical service system 200 may be variable. As an example only, the medical service device 240 may be connected to the processing device 210 via the network 220 or directly. For another example, the storage device 230 may be connected to the processing device 210 via the network 220 or directly.

[0089] The processing device 210 may process data and / or information obtained from the storage device 230, the medical service device 240, the perception device 250, the patient terminal device 260, and / or the doctor terminal device 270. For example, the processing device 210 may provide user services to the patient 261 and the doctor 271 through the patient terminal device 260 and / or the doctor terminal device 270, respectively. As another example, the processing device 210 may obtain data related to the patient 261 in the target medical scenario, and provide the target service by processing the perception information related to the target medical scenario and the data related to the patient 261.

[0090] In some embodiments, the processing device 210 may be configured with an agent. For example, the processing device 210 is configured to maintain a first agent corresponding to the medical service device 240, and implement user services related to the target medical scenario by enabling the first agent to participate in processing the perception information related to the target medical scenario collected by the perception device 250. For another example, the processing device 210 is configured to maintain a second agent corresponding to the perception device 250, and use the second agent to control the perception device 250 to collect perception information.

[0091] In some embodiments, the processing device 210 can be a single server or a server group. The server group can be centralized or distributed. In some embodiments, the processing device 210 can be located locally or remotely from the medical service system 200. In some embodiments, the processing device 210 can be implemented on a cloud platform. For example, a cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud cloud, a multi-cloud, etc., or a combination thereof. In some embodiments, the processing device 210 can be implemented by a computing device.

[0092] In some embodiments, the processing device 210 may include one or more processors (e.g., a single-core processor or a multi-core processor). For illustration purposes only, only one processing device 210 is described in the medical service system 200. However, it should be noted that the medical service system 200 in this specification may also include multiple processing devices. Therefore, as in this specification, the operations and / or method steps performed by one processing device 210 may also be performed jointly or individually by multiple processing devices. For example, if in this specification, the processing device 210 of the medical service system 200 simultaneously executes process A and process B, it should be understood that process A and process B may also be performed jointly or individually by two or more different processing devices in the medical service system 200 (e.g., the first processing device executes process A, the second processing device executes process B; or the first and second processing devices jointly execute processes A and B). In some embodiments, the processing device 210 is part of the medical service device 240. For example, the processing device 210 is integrated in the medical service device 240.

[0093] The network 220 may include any suitable network capable of facilitating information and / or data exchange of the healthcare service system 200. The network 220 may be or include a wired network, a wireless network (e.g., an 802.11 network, a Wi-Fi network), a Bluetooth TM network, near field communication (NFC) network, etc., or any combination thereof.

[0094] The storage device 230 may store data, instructions, and / or any other information. In some embodiments, the storage device 230 may store data obtained from other components of the medical service system 200. In some embodiments, the storage device 230 may store data and / or instructions that the processing device 210 may execute or use to execute the exemplary methods described in this specification.

[0095] In some embodiments, the data stored in the storage device 230 may include multimodal data. Multimodal data may include data in multiple forms (e.g., images, graphics, videos, text, etc.), various types of data, data obtained from different sources, data related to different medical services (e.g., diagnosis, surgery, rehabilitation, etc.), and data related to different users (e.g., patients, medical staff, managers, etc.). For example, the data stored in the storage device 230 may include medical data of the patient 261 reflecting the health status of the patient 261. For example, the medical data may include an electronic health record of the patient 261. An electronic health record refers to an electronic file that records various types of patient data (e.g., basic information, examination data, imaging data). For example, an electronic health record may include a three-dimensional model of multiple organs and / or tissues of the patient 261.

[0096] In some embodiments, the storage device 230 may include a mass storage device, a removable storage device, a volatile read-write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 230 may include a data lake and a data warehouse. For more information about the storage device, see Figure 3 Detailed description.

[0097] The medical service device 240 can be used to provide or assist medical services. Figure 2 As shown, the medical service equipment 240 includes a clinic terminal device 240-1, a hospital bed 240-2, an intelligent surgery terminal device 240-3, an intelligent nursing cart 240-4, an intelligent wheelchair 240-5, a registration terminal device 240-6, an intelligent mechanical nurse 240-7, etc., or any combination thereof.

[0098] The clinic terminal device 240-1 is also referred to as the first terminal device, and is configured in the clinic for use by doctors and patients in the medical consultation process. For example, the clinic terminal device 240-1 may include one or more of a screen, a sound output component, an image sensor, or a sound sensor. A consultation interface may be displayed on the screen of the clinic terminal device 240-1, and data may be displayed on the consultation interface to facilitate communication between doctors and patients. Exemplary data may include electronic health records (or a portion thereof), pre-consultation records, medical images, 3D organ models, test results, consultation recommendations, and the like. In some embodiments, the first terminal device is used in a target medical scenario, which is related to a medical consultation service provided to a patient in a clinic, and the first agent includes a consultation agent corresponding to the first terminal device, and at least a portion of the user services related to the target medical scenario are implemented based on the consultation agent.

[0099] A smart hospital bed 240-2 (or a meta-hospital bed) refers to a bed in a ward that can support inpatients and provide user services to patients. The smart hospital bed 240-2 may include a bed, a bedside terminal device (also referred to herein as a third terminal device), a bedside inspection device, a sensor, etc., or any combination thereof. The bedside terminal device may include an XR device, a display device, a mobile device, etc., or any combination thereof. In some embodiments, the smart hospital bed 240-2 is applied to a target medical scenario, which is related to the inpatient services provided to patients in the ward, the first agent includes an inpatient agent corresponding to the smart bed, and at least a portion of the user services related to the target medical scenario are implemented based on the inpatient agent.

[0100] The intelligent surgical terminal device 240-3 refers to a device used to assist surgery in an operating room. The intelligent surgical terminal device 240-3 can perceive the interaction (e.g., conversation, behavior, etc.) between the medical service provider, the patient, and the intelligent agent, and obtain the data captured by the sensing device 250, thereby providing surgical assistance. In some embodiments, the intelligent surgical terminal device 240-3 can be configured to perform risk warnings for surgery based on the intelligent agent configured therein, generate surgical records of the surgical process, and the like. In some embodiments, the intelligent surgical terminal device is applied to a target medical scenario, which is related to the surgical services provided to patients in the operating room, and the first intelligent agent includes a surgical intelligent agent corresponding to the intelligent surgical terminal device 240-3, and at least a portion of the user services related to the target medical scenario are implemented based on the surgical intelligent agent.

[0101] The smart nursing cart 240-4 refers to a nursing cart with an automatic driving function that can assist in the treatment and care of patients. For example, the smart nursing cart 240-4 can be configured to guide a nurse to the ward to perform a preliminary examination on the patient. In some embodiments, the smart nursing cart 240-4 may include a cart, a presentation device, one or more inspection devices and / or nursing tools, a sensing device (such as an image sensor, a GPS sensor, a sound sensor, etc.), etc. In some embodiments, the smart nursing cart 240-4 may be configured to obtain relevant treatment and nursing information of the patient and generate physical examination data, nursing data, etc. The physical examination data may include the patient's vital signs data. The nursing data may include a detailed record of the nursing operation, such as nursing time, nursing operator, nursing measures, patient reactions, etc. In some embodiments, the smart nursing cart 240-4 is applied to a target medical scenario, which is related to the inpatient service provided to the patient in the ward, the first agent includes a nursing agent corresponding to the smart nursing cart 240-4, and at least a part of the user service related to the target medical scenario is implemented based on the nursing agent.

[0102] Smart wheelchair 240-5 refers to a transportation device for intelligently picking up and dropping off patients. In some embodiments, smart wheelchair 240-5 can be configured to perform autonomous navigation through integrated sensors and maps (e.g., using simultaneous localization and mapping (SLAM) technology or a pre-built environmental model), locate the patient's position using a radio frequency identification device (RFID), Bluetooth or Wi-Fi signal, and identify the patient through biometric technology. In some embodiments, smart wheelchair 240-5 can be controlled by an intelligent agent (e.g., an intelligent agent corresponding to inpatient services, an intelligent agent corresponding to surgical services). In some embodiments, smart wheelchair 240-5 can be configured to generate data (e.g., a record of the content of the interaction between the intelligent agent and the patient) by sensing interaction data through a built-in camera / sensor.

[0103] The registration terminal device 240-6 refers to a terminal device installed on-site in a hospital for providing registration services, including a display screen, an input device (such as a keyboard), a sound output device, a sound sensor, an XR device, etc. or any combination thereof. In some embodiments, the registration terminal device 240-6 is applied to a target medical scenario, which is related to the registration service provided to the patient, the first agent includes a registration agent corresponding to the registration terminal device 240-6, and at least a part of the user service related to the target medical scenario is implemented based on the registration agent.

[0104] Intelligent mechanical nurse 240-7 refers to a robot used to assist in health care tasks and provide support to medical staff. In some embodiments, intelligent mechanical nurse 240-7 includes, for example, a display screen, a mechanical arm, a sensor (e.g., an image sensor, a GPS sensor, a sound sensor, etc.), etc. For example, intelligent mechanical nurse 240-7 is an embodied intelligent robot including one or more sensing devices and a processing device. In some embodiments, intelligent mechanical nurse 240-7 is applied to a target medical scenario, which is related to surgical services provided to patients in an operating room, and the first agent includes a mechanical nurse agent corresponding to intelligent mechanical nurse 240-7, and at least a portion of the user services related to the target medical scenario are implemented based on the mechanical nurse agent. For example, the mechanical nurse agent can control the intelligent mechanical nurse 240-7 to deliver target surgical tools to surgical participants, supplement surgical tools, etc.

[0105] In some embodiments, the medical service device 240 may further include a pre-consultation terminal device for providing pre-consultation services. For example, the pre-consultation terminal device may be installed in a waiting area near a consulting room and include a display screen, a sound output device, a sound sensor, an XR device, a wearable device, etc. or any combination thereof. In some embodiments, the pre-consultation terminal device is applied to a target medical scenario, which is related to the pre-consultation service provided to the patient, the first agent includes a pre-consultation agent corresponding to the pre-consultation terminal device, and at least a portion of the user service related to the target medical scenario is implemented based on the pre-consultation agent.

[0106] In some embodiments, the first agent corresponding to the medical service device can be configured on a processing device integrated into the medical service device. In some embodiments, the first agent corresponding to the medical service device can be configured to control the medical service device to perform specific operations so as to provide user services in the target medical scenario corresponding to the medical service device.

[0107] In some embodiments, the medical service device 240 includes a display component configured to display a virtual character representing the first agent. In some embodiments, at least a portion of the user service is implemented based on the interaction between the user of the target medical scenario and the virtual character. For more information about the interaction with the virtual character, see Figure 5 and its related description.

[0108] It should be noted that the above examples of medical service devices and their corresponding agents are for illustrative purposes only and are not limiting. In some embodiments, multiple medical service devices can share the same agent, for example, a robotic nurse agent and a surgical agent can simultaneously correspond to a surgical agent, and a smart bed and a smart nursing cart can simultaneously correspond to a hospitalization agent.

[0109] The sensing device 250 may be configured to collect sensing information related to the environment in which it is located. In some embodiments, the sensing device 250 may include a sensing device in the physical hospital 110. For example, the sensing device 250 may include an image sensor 250-1, a sound sensor 250-2, a temperature sensor, a humidity sensor, and the like.

[0110] The patient terminal device 260 may be a terminal device that interacts with the patient 261. In some embodiments, the patient terminal device 260 may include a mobile terminal device 260-1, an XR device 260-2, a smart wearable device 260-3, etc. The doctor terminal device 270 may be a terminal device that interacts with the doctor 271. In some embodiments, the doctor terminal device 270 may include a mobile terminal device 270-1, an XR device 270-2, etc. In some embodiments, the patient 261 may access a user space application (e.g., a patient space application) through the patient terminal device 260, and the doctor 271 may access a user space application (e.g., a medical space application) through the doctor terminal device 270. In some embodiments, the patient 261 and the doctor 271 may communicate with each other remotely through the patient terminal device 260 and the doctor terminal device 270, thereby providing telemedicine services, such as remote outpatient services, remote ward rounds services, remote follow-up services, etc.

[0111] The sensing device 250, the patient terminal device 260, and the doctor terminal device 270 may be configured as data sources to provide information to the medical service system 200. For example, these devices may transmit the collected data to the processing device 210, and the processing device 210 may provide user services based on the received data.

[0112] It should be noted that the description of the above-mentioned medical service systems 100 and 200 is intended to be illustrative, not to limit the scope of this specification. Many alternatives, modifications, and variations will be apparent to those skilled in the art. The features, structures, methods, and other features of the exemplary embodiments herein may be combined in various ways to obtain additional and / or optional exemplary embodiments. For example, the medical service system 200 may include one or more additional components, such as terminal devices of other users, public terminal devices of hospitals, and the like. As another example, two or more components of the medical service system 200 may be integrated into a single component.

[0113] Figure 3 is a schematic diagram of an exemplary hospital support platform 300 according to some embodiments of the present specification.

[0114] like Figure 3As shown, the hospital support platform 300 may include a hardware layer 310 (also referred to as a hardware module), an interface layer 320 (also referred to as an interface module), a data processing layer 330 (also referred to as a data processing module), an application development layer 340 (also referred to as an application development module), and a service layer 350 (also referred to as a service module). It should be understood that the "layers" and "modules" in this specification are only used to logically divide the components of the hospital support platform and are not intended to be limiting.

[0115] The hardware layer 310 can be configured to provide a hardware foundation for the interaction between the real world and the digital world, and can include one or more hardware devices related to hospital operations. Exemplary hardware devices can include medical service devices, sensor devices, terminal devices, and basic devices. For more information about hardware devices, see Figure 1 and its related description.

[0116] The interface layer 320 may be connected to the hardware layer 310 and the data processing layer 330. The interface layer 320 may be configured to obtain data collected by the hardware devices of the hardware layer 310 and send the data to the data processing layer 330 for storage and / or processing. The interface layer 320 may also be configured to control at least a portion of the hardware devices of the hardware layer 310. In some embodiments, the interface layer 320 may include a hardware interface and a software interface (e.g., a data interface, a control interface).

[0117] The data processing layer 330 may be configured to store and / or process data. The data processing layer 330 may include a processing device, and a plurality of data processing units may be configured on the processing device. The data processing layer 330 may be configured to obtain data from the interface layer 320 and process the data through at least one data processing unit to implement user services related to hospital business.

[0118] The data processing unit may include various preset algorithms for implementing data processing, which may be in the form of software, programs, computer codes, and / or instructions implemented by various computer programming languages ​​(e.g., Java, C / C++). In some embodiments, the data processing layer 330 may include a processing device (e.g., Figure 2 The data processing unit may be configured on the processing device. In some embodiments, the data processing unit may include an XR unit configured to process data using XR technology to implement XR services, an AI unit (e.g., an intelligent unit) configured to process data using AI technology to implement AI services, a digital twin unit configured to process data using digital twin technology to implement digital twin services, a data circulation unit configured to process data using data circulation technology (e.g., blockchain technology, data privacy computing technology) to implement data circulation services, etc.

[0119] In some embodiments, the data processing layer 330 may also include a data center configured to store data. In some embodiments, the data center may adopt a lake warehouse integrated architecture, which may include a data lake and a data warehouse. The data lake can be used to store large amounts of data in a tamper-proof manner. The data warehouse can be used to store index data corresponding to the data in the data lake. The data stored in the data lake may include native (or original) data collected by hardware devices, derived data generated based on native data, and the like. In some embodiments, the data in the data lake can be processed by a processing device (e.g., processing device 210).

[0120] The application development layer 340 may be configured to support application development, publishing, subscription, etc. The application development layer 340 is also referred to as the ecosystem layer. In some embodiments, the application development layer 340 may be configured to provide an open interface for application developers to access or call at least a portion of the data processing unit and use at least a portion of the data processing unit to develop applications. Figure 3 As shown, the application development layer 340 can provide development toolkits, application markets, multi-tenant operation platforms, cloud official websites, workspaces, and other supporting toolkits to assist developers in their work.

[0121] The service layer 350 can be configured so that users related to hospital services can access user services related to hospital services through user space applications. For more information about user services and user space applications, please refer to Figure 1 and its related description.

[0122] This specification provides a hospital support platform designed for the comprehensive management of various resources within the hospital, including hardware resources, software resources, and data resources. In some embodiments, the platform further integrates a data processing unit capable of supporting advanced technologies, such as artificial intelligence, XR, digital twins, and blockchain. These advanced technologies are used to improve the service efficiency and quality of the healthcare industry. For example, artificial intelligence technology enables the autonomous transformation and continuous optimization of hospital operations, while XR and digital twin technologies facilitate the creation and maintenance of virtual hospitals. The virtual hospital can interact with users to provide an immersive and novel service experience. In addition, the platform also includes an application development layer for granting access to these advanced technologies to third-party developers in the healthcare industry. This access promotes an open ecosystem, promotes the development and innovation of applications, and thus promotes the advancement of medical services.

[0123] Figure 4 2 is a block diagram of an exemplary processing device 210 according to some embodiments of the present specification. In some embodiments, the processing device 210 can be connected to a computer-readable storage medium (e.g., Figure 2The processing device 210 may include an acquisition module 410 and a service module 420.

[0124] The acquisition module 410 may be configured to collect data related to the target medical scenario (such as sensory information and patient-related data). For more information on acquiring data related to the target medical scenario, see Figure 1 , Figure 5 and its related description.

[0125] The service module 420 can be configured to provide a target service by processing data related to the target medical scenario. For more information about user services, see Figure 1 , Figure 5-Figure 7 and its related description.

[0126] In some embodiments, the processing device 210 may include one or more other modules. For example, the processing device 210 may include a storage module for storing data generated by modules in the processing device 210. In some embodiments, any two modules may be combined into a single module, and any one module may be divided into two or more units. For example, the service module 420 may include multiple units configured to support and / or implement different user services.

[0127] Figure 5 is a schematic diagram of an exemplary process 500 for providing user services according to some embodiments of the present specification.

[0128] like Figure 5 As shown, one or more perception devices 250 (eg, image sensor 250 - 1 , sound sensor 250 - 2 , etc.) may collect perception information 510 related to a target medical scenario.

[0129] The target medical scenario refers to a scenario where at least one medical service is provided. For example, the target medical scenario includes a hospital or a part of a hospital, such as a consulting room, a ward, a waiting room, an operating room, etc. In some embodiments, one or more sensing devices and medical service devices may be used in the target medical scenario.

[0130] In some embodiments, the processing device 210 may be configured to maintain a second agent corresponding to one or more sensory devices, and use the second agent to control the one or more sensory devices to collect sensory information. For example, the processing device 210 may use the second agent to control the image sensor 250-1 to collect image data related to the target medical scene, and control the sound sensor 250-2 to collect sound data related to the target medical scene.

[0131] In some embodiments, the processing device 210 may obtain the perception information 510. For example, one or more perception devices 250 may send the perception information 510 to the processing device 210. For another example, one or more perception devices 250 may store the perception information 510 in a storage device (e.g., storage device 230), and the processing device 210 may retrieve the perception information 510 from the storage device.

[0132] In some embodiments, the processing device 210 can provide user services 540 related to the target medical scenario by processing the perception information 510. For example, the processing device 210 can be configured to maintain a first agent 520 corresponding to each medical service device used in the target medical scenario, and implement at least a portion of the user services related to the target medical scenario by enabling the first agent 520 to participate in processing the perception information. For example, the processing device 210 can display a virtual character 534 corresponding to the first agent 520 through the display component 530 of the medical service device, and can provide user services 540 based on the interaction between relevant users of the target medical scenario (e.g., patients, hospital staff (e.g., doctors, nurses, hospital administrators)) and the virtual character 534. In some embodiments, relevant users can access the target medical scenario through user terminal devices (e.g., Figure 5 The XR device 532 shown in FIG. 5 interacts with the virtual character 534 .

[0133] A virtual character refers to a computer-generated person or entity designed to interact with a relevant user in a digital environment. The virtual character 534 can be configured to interact with the relevant user to provide user services 540. For example, the virtual character 534 can communicate with the user by simulating human voice expressions, gestures, etc., to provide the user with a realistic communication experience. During the communication process, the user can use natural language to express his / her needs or requests through voice, text, gestures, etc., and the first agent 520 can be built based on a natural language processing algorithm to understand and analyze the user's input, determine feedback information, and convey the feedback information to the user through the virtual character 534.

[0134] In some embodiments, the virtual character 534 may be a digital character having certain appearance features, voice features, etc. For example, the processing device 210 may determine the appearance features and / or voice features of the virtual character 534 based on the patient data of the patient. As another example, the processing device 210 may determine the appearance features and / or voice features of the virtual character 534 based on the data of the patient's doctor or nurse, so that the virtual character 534 can simulate the doctor or nurse.

[0135] In some embodiments, the processing device 210 is further configured to obtain patient data 560 related to the patient in the target medical scenario. The patient data 560 includes multimodal data. Multimodal data includes data in various forms (such as images, graphics, videos, text, etc.), various types of data, data obtained from different sources, etc. For example, the patient data 560 may include electronic health records, interaction information related to the interaction between the patient and the patient terminal device, the patient's physical examination data (such as examination reports, medical images, vital signs), medical orders, the patient's clinical pathway, data related to the patient input by a medical service provider (such as a doctor, a nurse), etc. or any combination thereof. For another example, the patient data 560 can be collected from one or more hardware devices, such as a storage device of a hospital (such as a target medical scenario), a patient's terminal device of a patient, a terminal device of a medical service provider, an inspection device (such as a vital sign monitor, a medical imaging device, a wearable inspection device), a public terminal device of a hospital, etc. or any combination thereof.

[0136] In some embodiments, the processing device 210 may collect patient data 560 based on a preset data collection protocol corresponding to the target medical scenario. For example, at least one hardware device may collect patient data 560 related to the patient based on a preset data collection protocol, and the processing device 210 may obtain the patient data 560 from the at least one hardware device.

[0137] In some embodiments, the processing device 210 may provide user service 540 by engaging the first agent 520 in processing the sensory information 510 and the patient data 560 .

[0138] Exemplarily, user services 540 may include medical consultation services, registration services, pre-consultation services, hospitalization services, admission inquiry services, ward rounds services, nursing services, path guidance services, discharge services, follow-up services, etc. or any combination thereof. Some of these services may be provided by different agents. For example, the registration service may be provided by the registration agent corresponding to the registration terminal device based on the interaction between the patient and the nurse virtual character representing the registration agent. For another example, the admission inquiry service may be provided by the hospitalization agent corresponding to the smart bed based on the interaction between the patient and the doctor virtual character representing the registration agent.

[0139] In some embodiments, the first agent 520 may generate a processing result 550 based on the perception information 510 and / or the patient data 560, and may provide user services 540 to relevant users based on the processing result 550. For example, the processing result 550 may include a speech recognition result, an image recognition result, an abnormality detection result, etc. The first agent 520 may generate the processing result 550 using various artificial intelligence technologies (such as a machine learning model). For example, the relevant users include hospital staff (such as doctors, nurses, and hospital managers), and the user services 540 provided to the hospital staff include providing records, notifications, suggestions, etc. based on the processing result 550.

[0140] In some embodiments of this specification, by integrating intelligent agents, user services can be automatically provided, thereby reducing the need for a large amount of manpower, reducing operating costs, and improving the efficiency of providing user services. In addition, the intelligent agent can also continuously optimize the learned rules and mechanisms to improve the accuracy, efficiency and service quality of user services. In addition, this specification also provides different types of intelligent agents that can handle tasks of different medical service providers, different departments, different hospital positions, etc. Each intelligent agent can achieve self-optimization and evolution from specific data to better handle the corresponding tasks.

[0141] Figure 6 FIG. 6 is a schematic diagram of a medical consultation process 600 according to some embodiments of the present specification. Figure 6 As shown, the medical consultation process 600 includes multiple stages, such as a registration stage 610, a waiting stage 620, a consultation stage 630, and a post-consultation stage 640. In some embodiments, the processing device 210 (e.g., the service module 420, an intelligent agent corresponding to a medical service device used in a consultation service / process configured on the processing device 210) can perform operations involved in multiple stages of the medical consultation process 600.

[0142] Patients can make an appointment to see a doctor at 610 during the registration process. Figure 6 As shown, operations related to the registration link 610 may include obtaining / establishing an electronic health record, providing intelligent registration services, providing path planning services, providing path guidance services, etc. In some embodiments, a patient may initiate a registration request through a patient terminal device or a registration terminal device at a hospital site, or initiate a remote registration request through a patient terminal device at other locations outside the hospital.

[0143] In some embodiments, after receiving a registration request from a patient, the processing device 210 may obtain or establish an electronic health record of the patient. An electronic health record refers to an electronic record that records various types of patient data. In some embodiments, the electronic health record may be updated as the patient consultation process progresses. For example, when a patient registers, the electronic health record may include the patient's basic information; after the patient completes the registration, the electronic health record may be updated to further include the patient's chief complaint and registration record.

[0144] In some embodiments, when receiving a registration request from a patient, the processing device 210 may provide the patient with an intelligent registration service. The intelligent registration service may be used to match the patient with a registration department, a registration doctor, and a registration time. In some embodiments, the processing device 210 may provide services related to the registration process through a registration terminal device of the hospital or a patient terminal device of the patient. In some embodiments, the processing device 210 may perform Figure 7 Process 700 in determines the doctor with whom the patient is registered.

[0145] The route planning service can be used to provide the patient with a planned route to the registered doctor's office. For example, the processing device 210 can generate a planned route to the office based on the patient's current location and the location information of the registered doctor's office. In some embodiments, the processing device 210 can generate a planned route after the patient registers on-site at the hospital. In some embodiments, the processing device 210 can generate a planned route when the patient registers remotely and arrives at the hospital.

[0146] The path guidance service can be used to guide the patient to the registered doctor's office according to the planned path. In some embodiments, the processing device 210 can display guidance information to the patient through the patient terminal device (e.g., XR device) of the patient. For example, through AR technology, the XR device can overlay the guidance information related to the planned path on the patient's real view.

[0147] The patient may wait and prepare for the consultation in the waiting section 620. In the waiting section 620, the patient may be on-site at the hospital (e.g., in the waiting area in front of the registered doctor's office) or at another location outside the hospital (e.g., at the patient's home). Figure 6 As shown, operations related to the waiting stage 620 may include providing pre-consultation services, providing force feedback / temperature feedback, etc.

[0148] The pre-consultation service can be used to collect information about the patient by conducting preliminary inquiries to the patient before the patient enters the consulting room for formal consultation. For example, when the patient is waiting for consultation, the processing device 210 can conduct a pre-consultation to the patient through the patient terminal device of the patient or the waiting terminal device configured in the waiting area to relieve the patient's anxiety while waiting, generate a pre-consultation record, and provide the pre-consultation record to the doctor for reference, thereby improving the doctor's consultation efficiency.

[0149] Force feedback / temperature feedback can be used to provide comforting feedback to patients. For example, when a patient is waiting for a consultation, after the processing device 210 detects the patient's emotions (e.g., nervousness, fear, anxiety, etc.), it can apply force feedback and / or temperature feedback to the patient through the patient's wearable device, so that the patient can feel the actions of handshake, hug, etc., thereby comforting the patient's negative emotions.

[0150] The patient can communicate with the registered doctor in the consultation session 630 to receive medical outpatient services (e.g., on-site outpatient services in the clinic, remote outpatient services). Figure 6 As shown, operations related to the consultation stage 630 may include preview of the day's clinic, display of medical data, provision of consultation advice, generation of diagnosis records, remote accompanying services, etc.

[0151] In the consultation link 630, the clinic preview of the day is used to show the doctor the relevant information of the patients to be consulted on the day. For example, the processing device 210 can show the doctor the categories of patients who are registered or waiting for consultation on the day before the doctor starts to see the doctor. The categories of patients can include first-visit patients, return patients, etc. In some embodiments, the doctor can browse the clinic preview of the day through the first terminal device in the consulting room or the doctor's terminal device (e.g., XR device).

[0152] In the consultation link 630, the medical data display is used to display the patient's medical data, such as an electronic health record, to the doctor, the patient and / or the remote accompanying person. For example, the processing device 210 can synchronously display the patient's medical data to the doctor, the patient and / or the remote accompanying person through at least one terminal device. In addition, the display method and / or display content of the electronic health record are updated according to the interactive operation of the doctor, the patient and / or the remote accompanying person on the medical data, so that the doctor, the patient and / or the remote accompanying person can communicate.

[0153] In the consultation link 630, providing consultation suggestions refers to providing the doctor with consultation suggestions that can be used as a reference during the consultation process. For example, the processing device 210 can generate consultation suggestions for the doctor based on the sensing information collected by the sensing device during the consultation process, so that the doctor can adjust the diagnosis method, diagnosis result and prescription, thereby improving the consultation efficiency and accuracy.

[0154] The diagnostic record generation is used to assist doctors in generating diagnostic records for patients. For example, the processing device 210 can generate a diagnostic record to record information about the patient's medical condition diagnosed by a doctor, the doctor's medical advice, etc.

[0155] The remote medical companion service can be used to provide an immersive medical companion service for patients and remote medical companions. For example, the processing device 210 can show the remote medical companion a virtual medical room that simulates a real medical room, a real-time picture of the doctor in the medical room, and a real-time picture of the patient. As another example, the processing device 210 can present a real-time image of the remote medical companion through the first terminal device in the medical room. As another example, the processing device 210 can present the real-time picture of the remote medical companion to the doctor and the patient through the doctor's doctor terminal device and the patient's patient terminal device, respectively.

[0156] For more information about medical data display, providing consultation advice, generating diagnosis records and remote accompanying services, please refer to Fig.10 The relevant description will not be repeated here.

[0157] In some embodiments, the processing device 210 can provide services related to the consultation process to relevant users (e.g., doctors, patients, and remote accompanying persons) through at least one terminal device. The at least one terminal device includes a first terminal device in the clinic, a patient terminal device, a doctor terminal device, a remote accompanying person terminal device, etc.

[0158] The first terminal device in the clinic refers to a terminal device installed on-site in the clinic, which may include a display screen, a sound output device, a sound sensor, an XR device, a wearable device, etc., or any combination thereof. For example, after the patient enters the clinic, the processing device 210 may present electronic health records, consultation recommendations, pre-consultation records, diagnosis records, real-time images of remote accompanying persons, etc. to the doctor and / or the patient through the display screen of the first terminal device. For another example, the patient may wear a wearable device of the first terminal device and receive force feedback from the remote accompanying person through the wearable device. For another example, the doctor and the patient may wear an XR device of the first terminal device, and the processing device 210 may present various types of information to the doctor and / or the patient through the XR device of the first terminal device. In some embodiments, the XR device worn by the patient may also be an XR device of the registration terminal device.

[0159] refer to Fig.11 , Fig.11 is a schematic diagram of an exemplary consultation interface 1100 according to some embodiments of this specification. Fig.11As shown, the consultation interface 1100 may include a first interface element 1110 related to the patient's electronic health record, a second interface element 1120 related to the remote accompanying service, a third interface element 1130 related to the medical document, and a fourth interface element 1140 related to the consultation recommendation.

[0160] The presentation content and / or presentation form of the consultation interface 1100 may change as the consultation process proceeds. For example, the doctor and / or the patient may issue a control instruction to adjust the content displayed in the consultation interface 1100. The control instruction may include various forms of instructions, such as voice control instructions, gesture control instructions, touch control instructions, and control instructions input through an input device (e.g., a mouse, keyboard, etc.).

[0161] The first interface element 1110 may be an icon corresponding to an electronic health record, or may be used to display the content of the electronic health record. During the consultation process, the doctor and / or the patient may retrieve at least a portion of the electronic health record by issuing a control instruction to present it through the first interface element 1110. For example, in response to detecting that the patient has started a consultation, the processing device 210 may display an interface element (e.g., the first interface element 1110) related to the patient's electronic health record through a first terminal device (e.g., a clinic terminal device).

[0162] The second interface element 1120 may be an icon corresponding to the remote medical companion service, or may be used to display a remote medical companion screen. When the patient's remote medical companion request is approved by the doctor, the patient may communicate with the remote medical companion during the consultation process and view the remote medical companion screen through the second interface element 1120.

[0163] The third interface element 1130 may be an icon corresponding to a medical document, and may also be used to display a medical document, such as a pre-consultation record, a diagnosis record (initial diagnosis record or target diagnosis record), etc. When it is detected that the patient has started a consultation, the pre-consultation record may be presented through the third interface element interface element 1130; when the doctor and the patient have completed the communication, the third interface element interface element 1130 may be updated to present the initial diagnosis record; when the doctor confirms the modification of the initial diagnosis record, the third interface element interface element 1130 may be updated to present the target diagnosis record.

[0164] In some embodiments, the processing device 210 can determine that the patient starts the consultation based on the calling instruction. The calling instruction is an instruction from the doctor to start the consultation with the next patient. The doctor can actively initiate the calling instruction through a public terminal device or a doctor terminal device. Alternatively, after the doctor submits the target diagnosis record of the previous patient, the processing device 210 can automatically generate a calling instruction. In some embodiments, when the patient receives on-site medical outpatient services, the processing device 210 can determine that the patient starts the consultation based on the positioning information of the patient's terminal device. Alternatively, the processing device 210 can obtain an image of the patient entering the consulting room based on a sensing device (e.g., an image sensor) to determine that the patient starts the consultation. When the patient receives telemedicine outpatient services, the processing device 210 can obtain the remote consultation confirmation messages sent by the doctor and the patient through the doctor's terminal device and the patient's terminal device, respectively, to determine that the patient starts the consultation.

[0165] The fourth interface element 1140 may be an icon corresponding to the consultation suggestion, or may be used to present the content of the consultation suggestion. During the consultation process, after the consultation suggestion is generated based on the perception information, the consultation suggestion may be presented through the fourth interface element 1140 .

[0166] In some embodiments, the processing device 210 may provide services related to the consultation process to the doctor and the patient through the doctor's doctor terminal device and the patient's patient terminal device. At least a portion of the content presented by the doctor's terminal device and the patient's terminal device may be synchronized.

[0167] In some embodiments, the processing device 210 can further provide the remote patient with services related to the consultation process through the remote patient's remote terminal device (e.g., XR terminal device). When the remote patient participates in the consultation process, the remote patient's remote terminal device can synchronously present the virtual consulting room.

[0168] In some embodiments, the processing device 210 may provide telemedicine outpatient services. For example, the processing device 210 may obtain or generate a three-dimensional patient model. The 3D patient model may correspond to a patient or a part of a patient (e.g., the upper body). As an example only, the processing device 210 may obtain an initial three-dimensional patient model from the patient's electronic health record, and may update the initial three-dimensional patient model based on the patient's real-time dynamic data and physiological data, thereby obtaining a three-dimensional patient model. In addition, the processing device 210 may present the three-dimensional patient model to the doctor, and obtain the examination instruction input by the doctor through the doctor's terminal device. For example, the processing device 210 may present the three-dimensional patient model in the doctor's field of view through the doctor's terminal device. The examination instruction input by the doctor may include an examination part, an examination device, and an examination operation. The doctor may input the examination instruction in various ways, such as voice, gesture, and operation input device (e.g., smart gloves, smart grips, etc.). For example, the doctor's terminal device may present a virtual examination device corresponding to a plurality of examination devices. The doctor may select a virtual examination device through an input device, and perform a virtual examination operation on the three-dimensional patient model using the virtual examination device. The processing device 210 can determine the examination part, examination equipment, examination operation, etc. according to the virtual examination operation performed by the doctor, and generate an examination instruction. In addition to the examination instruction, the doctor can also input other instructions to instruct the doctor terminal device to rotate, enlarge, reduce, etc. the three-dimensional patient model.

[0169] In some embodiments of this specification, doctors and patients can communicate remotely in a virtual consulting room space through a doctor terminal device and a patient terminal device, which can provide doctors and patients with an unimpeded immersive consultation experience. In addition, a three-dimensional patient model of the patient can be presented to the doctor, so that remote examinations can be performed more accurately and conveniently, thereby improving the accuracy of diagnosis.

[0170] After the patient completes the consultation, he or she may proceed to the post-consultation phase 640. Figure 6 As shown, operations related to the post-diagnosis stage 640 may include providing medication pickup services, providing examination services, providing health monitoring services, etc.

[0171] The medication pickup service is used to assist patients in getting the medications prescribed by doctors. For example, the medication pickup service is used to assist patients in paying medication fees, making appointments with pharmacies to pick up medications, and guiding patients to pharmacies. The examination service is used to assist patients in receiving examinations requested by doctors. For example, the examination service is used to assist patients in paying examination fees, making appointments with examination departments, and guiding patients to examination departments.

[0172] In some embodiments, in response to detecting that the consultation process is over, the processing device 210 may determine the target service to be provided to the patient after the consultation process, and make an appointment for the patient with the target business unit that provides the target service. The processing device 210 may detect whether the consultation process is over in a variety of ways. For example, when it is detected that the doctor has submitted the patient's target diagnosis record or the doctor calls the next patient, the processing device 210 may determine that the patient's consultation process is over.

[0173] In some embodiments, the processing device 210 can determine the target service required by the patient based on the target treatment prescription in the target diagnosis record. For example, the target service can be a medication pickup service, and the target business unit can be a pharmacy. When the patient needs a medication pickup service, the processing device 210 can send the patient's medication prescription information to the pharmacy and make an appointment to pick up the medication. For another example, the target service can be an examination service, and the target business unit can be an examination department. When the patient needs an examination service, the processing device 210 can send the patient's examination prescription information to the examination department and make an appointment for the examination. For example, after making an appointment with the target business unit, the processing device 210 can send the appointment information to the patient's terminal device.

[0174] The health monitoring service can be used to continuously monitor the patient's health status after the consultation process. For example, the processing device 210 can generate a health monitoring plan based on the target diagnosis record, and enable one or more monitoring devices to obtain the patient's health monitoring information based on the health monitoring plan. Optionally, the processing device 210 can further update the health monitoring plan based on the health monitoring information.

[0175] In some embodiments, one or more operations in the medical consultation process 600 can be performed by an agent corresponding to the medical service equipment used in the medical clinic service / process. In some embodiments, different links of the medical consultation process 600 can share an agent, or different links can correspond to different agents.

[0176] Figure 7 is an exemplary schematic diagram of determining the doctor who registers a patient according to some embodiments of the present specification. Figure 7 The process 700 shown in can be executed in a target medical scenario, which is related to the registration service provided to the patient in the registration link 610.

[0177] Step 710, obtaining the patient's chief complaint.

[0178] The patient's chief complaint refers to the patient's self-report of the condition and / or symptoms. In some embodiments, the patient's chief complaint may be in the form of, but not limited to, text, voice, picture, gesture, etc. In some embodiments, the processing device 210 may obtain the patient's chief complaint through the patient's terminal device or the hospital's registration terminal device.

[0179] Step 720, determining at least one candidate department based on the patient's chief complaint.

[0180] A candidate department refers to a department that is preliminarily determined based on the patient's condition and / or symptoms. For example, the processing device 210 may extract the first keyword of the patient's main complaint. The first keyword may include key words that summarize the condition and / or symptoms, such as diseased site, symptom, time, intensity, frequency, etc. In some embodiments, the processing device 210 may determine at least one candidate department based on the first keyword. For example, the processing device 210 may determine at least one candidate department by retrieving a preset keyword-department comparison table. For another example, the processing device 210 may input the first keyword into a department determination model to determine at least one candidate department. The department determination model is a pre-trained machine learning model that can process the model input to output a recommended candidate department. In some embodiments, the preset keyword-department comparison table and / or department determination model can be learned by the intelligent agent corresponding to the medical consultation process from historical registration records.

[0181] Step 730: Control the patient terminal device or the registration terminal device to make a first inquiry to the patient based on at least one candidate department.

[0182] The first inquiry is used to further clarify the patient's consultation needs, so as to determine a matching registration department and / or registration doctor for the patient. The first inquiry may also be referred to as a registration inquiry. In some embodiments, the first inquiry may include multiple rounds of dialogue. Each round of dialogue may include one inquiry and one patient answer.

[0183] In some embodiments, the processing device 210 may determine the first inquiry content of the first inquiry based on the patient's chief complaint and at least one candidate department. The first inquiry content includes at least the content of the first round of inquiries. For example, the processing device 210 may input the patient's chief complaint and at least one candidate department into the first inquiry model, and the first inquiry model outputs the content of the first round of inquiries. In some embodiments, for subsequent rounds of inquiries in the first inquiry, the processing device 210 may use a second inquiry model to determine the corresponding first inquiry content based on the patient's chief complaint, at least one candidate department, and historical rounds of dialogue.

[0184] In some embodiments, the processing device 210 can make the first inquiry by causing the patient terminal device or the registration terminal device to display or play the first inquiry content. In some embodiments, the processing device 210 can cause the patient terminal device or the registration terminal device to display the first virtual character that makes the first inquiry. The first virtual character is a digital character image with specific appearance characteristics, tone characteristics, etc., which can communicate with the patient and assist the patient in registration. In some embodiments of the present specification, using the first virtual character that can communicate with the patient in natural language to make the first inquiry can enhance the patient's sense of interaction and improve the quality and efficiency of the registration service.

[0185] Step 740, based on the first data collected in the first inquiry, determine the doctor that the patient wants to register with.

[0186] In some embodiments, the first data includes perceptual information related to the registration service that can be collected by one or more perceptual devices in the target medical scenario. Perceptual information includes voice data, text data, image data, etc. Further, the processing device 210 can determine at least one recommended department based on the confidence of at least one candidate department, so that the patient can select at least one registration department based on at least one recommended department. After the registration department is determined, the patient terminal device or the registration terminal device can display the registration link corresponding to the registration department, and the patient further selects the registration doctor and registration time by clicking on the registration link.

[0187] In some embodiments, the processing device 210 can display the basic information and available registration time of each doctor corresponding to the registration department through the patient terminal device or the registration terminal device. As an example only, the processing device 210 can cause the XR device of the patient terminal device or the registration terminal device to display a virtual character of the doctor, and the virtual character can introduce the basic information and available registration time of the doctor. In some embodiments of the present specification, the first inquiry is to communicate with the patient so as to fully understand the patient's consultation needs. This can enhance the accuracy of the registration suggestions provided to the patient, improve the efficiency of the registration process, and enhance the patient's interactive experience at the same time.

[0188] In some embodiments, at least a portion of process 700 may be performed by an agent corresponding to the registration terminal device (also referred to as a registration agent). The registration agent can understand and process natural language and images, allowing patients to describe symptoms in their own language and accurately assess the patient's condition, thereby providing high-confidence registration recommendations. At the same time, the registration agent can continuously improve the accuracy and efficiency of the intelligent registration service by learning knowledge data and historical service data, thereby improving the user experience.

[0189] Figure 8It is an exemplary schematic diagram of providing pre-consultation services according to some embodiments of the present specification. Figure 8 The process 800 shown in can be executed in a target medical scenario, which is related to the pre-consultation service provided to the patient in the waiting stage 620. In some embodiments, at least a part of the process 800 is executed by a pre-consultation agent.

[0190] Step 810, determining the second inquiry content of the second inquiry based on the doctor's department.

[0191] The second inquiry may also be referred to as a pre-inquiry inquiry, which is used to conduct preliminary inquiries to the patient before the formal inquiry. The second inquiry may include multiple rounds of inquiries. The second inquiry content may include the inquiry content of each round of inquiries. Alternatively, the second inquiry content only includes the inquiry content of the first round of inquiries. The second inquiry content may also be referred to as a pre-inquiry inquiry content.

[0192] In some embodiments, the processing device 210 may obtain a pre-consultation record template corresponding to the doctor's department, and determine the second inquiry content according to the pre-consultation record template.

[0193] In some embodiments, the processing device 210 can obtain known information about the patient (e.g., electronic health records, chief complaints, etc.), and determine the missing information that has not been collected in the pre-consultation record template by comparing the pre-consultation record template with the known information. For example, if the known information includes the patient's family medical history, the missing information does not need to include the family medical history. For another example, if the patient's chief complaint includes the patient's medical history, the missing information does not need to include the medical history. In some embodiments, the missing information can be determined based on the patient's basic information. For example, for male patients, the omission does not need to include menstrual history and reproductive history. In addition, the processing device 210 can determine the second inquiry content based on the missing information.

[0194] In some embodiments, the processing device 210 may use a third inquiry model to determine the second inquiry content based on the doctor's department and known information about the patient. The third inquiry model may include a CNN model, an RNN model, an LSTM model, a BERT model, a ChatGPT model, etc. In some embodiments, the third inquiry model may include a missing information determination model and a first inquiry content determination model. The missing information determination model may be configured to output missing information by processing the doctor's department and known information about the patient. The first inquiry content determination model may be configured to output the second inquiry content based on the patient's missing information.

[0195] Step 820: Based on the second inquiry content, control the pre-inquiry terminal device or the patient terminal device to conduct a second inquiry to the patient.

[0196] In some embodiments, after the patient registers with the doctor, the processing device 210 can determine the estimated waiting time for the patient to receive medical outpatient services. For example, the estimated waiting time can be the time difference between the current moment and the patient's registration time period. For another example, the estimated waiting time can be determined based on the doctor's outpatient record of the day and the patient's registration record. The patient's registration record may include the registration time period reserved by the patient. The doctor's outpatient record of the day is a record reflecting the doctor's outpatient situation on the day.

[0197] In some embodiments, in response to determining that the estimated waiting time is greater than the first preset time threshold, the processing device 210 can cause the pre-consultation terminal device or the patient terminal device of the patient to initiate a second inquiry to the patient or display a suggestion for a second inquiry. This approach can ensure that there is sufficient time for the pre-consultation and prevent the doctor from calling the patient during the pre-consultation process.

[0198] In some embodiments, in response to determining that the estimated waiting time is less than a second preset time threshold, the processing device 210 may cause the pre-consultation terminal device or the patient's terminal device to initiate a second inquiry to the patient or display a suggestion for a second inquiry. The second preset time threshold may be greater than the first preset time threshold. For example, when it is detected that the time from the current moment to the registration time period is less than 24 hours (i.e., the estimated waiting time is less than 24 hours), the pre-consultation terminal device or the patient terminal device may display a suggestion for a second inquiry to the patient (such as making a suggestion to the patient through a virtual character), thereby promptly reminding the patient to conduct a pre-consultation.

[0199] In some embodiments, the processing device 210 may detect that the patient initiates a pre-consultation request through the pre-consultation terminal device or the patient terminal device, and then enable the pre-consultation terminal device or the patient terminal device to perform a second inquiry on the patient.

[0200] In some embodiments, the pre-consultation terminal device or the patient terminal device may present a second virtual character to conduct a second inquiry based on the second inquiry content. The second virtual character refers to a digital character image with specific characteristics (e.g., specific appearance characteristics, sound characteristics, etc.), and can communicate with the patient to conduct a pre-consultation of the patient. For example, the processing device 210 may display the second virtual character through the screen of the pre-consultation terminal device or the patient terminal device (e.g., an XR device), and play the second inquiry content through the sound output device of the pre-consultation terminal device or the patient terminal device. At the same time, the second virtual character can simulate human voice expressions, gestures, etc. to provide patients with a realistic communication experience.

[0201] In some embodiments, the second virtual character may have a preset appearance feature. In some embodiments, the appearance feature of the second virtual character may be determined based on the optical image data of the doctor who registered the patient. In some embodiments, the appearance feature of the second virtual character may be determined based on the basic information of the patient. In some embodiments, the processing device 210 may select a suitable virtual character from the virtual character library as the second virtual character based on the appearance feature of the doctor and / or the basic information of the patient.

[0202] In some embodiments, the second inquiry includes multiple rounds of inquiry, and the second inquiry content may include the inquiry content of each round of inquiry in the second inquiry. Fig. 9 The process 900 shown is performed.

[0203] like Fig. 9 As shown, for the first round of inquiries, the processing device 210 may enable the pre-inquiry terminal device or the patient terminal device to perform the first round of inquiries based on the corresponding inquiry content.

[0204] For each current round of inquiries (referred to as the current inquiry for short) except the first round of inquiries, the processing device 210 can adjust the inquiry content of the current inquiry (referred to as the current inquiry content for short) based on the second data collected before the current inquiry, so that the inquiry content is more in line with the patient's condition. For example, the processing device 210 can determine the semantic information and emotional information of the patient's historical answers based on the second data collected before the current inquiry. The second data may include voice signals, image data, text data, etc. collected by the pre-inquiry terminal device or the patient terminal device. The historical answer is the patient's answer to the historical round of inquiries. The semantic information of the historical answer represents the content of the historical answer. The emotional information of the historical answer can indicate the patient's emotions when providing the historical answer (for example, calm, nervous, anxious, afraid, suspicious, irritable, etc.). The processing device 210 can determine the semantic information by performing text transcription, voice content recognition, etc. on the second data. The processing device 210 can determine the emotional information by analyzing the content, tone, intonation, speech speed and other features of the second data.

[0205] Continue to refer Fig. 9 , the processing device 210 can adjust the current inquiry content based on semantic information and emotional information. For example, when the patient's emotional information is "nervous" or "afraid", the processing device 210 can add soothing words to the current inquiry content. For another example, when the semantic information indicates that the patient has not clearly answered the historical inquiry, the processing device 210 can adjust the current inquiry content to repeat the historical inquiry, thereby guiding the patient to clearly answer the historical inquiry. The originally determined current inquiry content can be used as the inquiry content of the next round of inquiries. In this way, the current consultation content can be adjusted in time according to the patient's condition, thereby improving the service quality of the pre-consultation.

[0206] In some embodiments, in addition to adjusting the current inquiry content, the sound features used for the inquiry can also be adjusted in real time based on the patient's condition. Sound features include speech speed features, tone features, intonation features, volume features, etc. Fig. 9 As shown, the processing device 210 can determine the voice characteristics of the current inquiry based on the semantic information and emotional information of the patient's historical answers, and make the pre-consultation terminal device or the patient terminal device perform the current inquiry based on the adjusted inquiry content and the voice characteristics of the current inquiry. This method can better take care of the patient's emotional changes, thereby enhancing the anthropomorphic effect of the second virtual character and improving the quality of the pre-consultation service.

[0207] In some embodiments, Fig. 9 As shown, the processing device 210 can further obtain the patient's physiological state information. The patient's physiological state information can reflect the patient's real-time physiological state. The physiological state information can include the patient's physiological parameter values ​​(e.g., heart rate, pulse rate, respiratory rate, etc.). The physiological state information can also include information related to the patient's posture, limb behavior, facial expression, muscle state, etc. In some embodiments, the patient's physiological state information can be obtained using a wearable device worn by the patient.

[0208] In addition, the processing device 210 can adjust the current query content based on the semantic information, the emotional information and the physiological state information. For example, the processing device 210 can update the patient's emotional information based on the patient's physiological state information. It is understandable that the patient's inner emotions are not always fully expressed through the patient's answers, so the patient's emotional information can be updated or modified based on the patient's physiological state information. In addition, the processing device 210 can adjust the current query content based on the semantic information and the updated emotional information.

[0209] According to some embodiments of the present specification, by further considering the patient's physiological state data, the accuracy of the patient's emotional information can be improved, thereby improving the accuracy of the current inquiry content adjustment, thereby improving the service quality of the pre-consultation service.

[0210] like Fig. 9As shown, in some embodiments, the processing device 210 can determine feedback parameters based on at least a portion of the semantic information, emotional information, and physiological state information, and control the wearable device to apply feedback to the patient based on the feedback parameters. The feedback may include at least one of force feedback or temperature feedback. The feedback parameters can be used to control the manner in which feedback is applied, such as the type of feedback, the part of the body to which feedback is applied, the intensity of feedback, and the like. In some embodiments, the processing device 210 can determine the patient's emotions and emotional levels based on at least a portion of the semantic information, emotional information, and physiological state information, and determine the feedback parameters based on the emotions and emotional levels. This method can soothe the patient's negative emotions in a timely manner, thereby improving the quality of pre-consultation services.

[0211] In some embodiments, the processing device 210 may end the second inquiry based on a second preset condition. The second preset condition may be that the number of remaining missing information is 0. The second preset condition may be that the time difference between the patient's current time and the estimated waiting time is less than a threshold. The second preset condition may also be similar to the first preset condition.

[0212] In some embodiments, the second inquiry content determined in step 810 may only include the inquiry content of the first round of inquiry. The current inquiry content of each current inquiry except the first round of inquiry may be determined during the second inquiry. For example, in the current inquiry, the processing device 210 may input the inquiry content of the historical inquiry, the patient's historical answers, the patient's known information, etc. into the second inquiry content determination model, and the second inquiry content determination model outputs the current inquiry content.

[0213] Step 830: Generate a pre-inquiry record based on the second data collected in the second inquiry.

[0214] In some embodiments, the second data includes sensory information, which can be collected by one or more sensory devices in the target medical scenario related to the pre-consultation service. The sensory information may include voice data, text data, image data, etc. The pre-consultation record can be used to record the patient information collected in the second consultation (i.e., pre-consultation). Optionally, some known information of the patient can also be recorded in the pre-consultation record. In some embodiments, the pre-consultation record is generated according to a preset template. The preset template can be a template corresponding to the doctor's department, or it can be a template set by the doctor.

[0215] For example, when the second data includes a voice signal, the processing device 210 may first transcribe the voice signal into text, and extract the second keyword from the text through a keyword extraction algorithm. Then, the processing device 210 may convert the second keyword into a medical term. In addition, the processing device 210 may obtain multiple template fields in the pre-consultation record template, retrieve the content corresponding to each template field from the hospital terminology, and fill it in the corresponding position of the pre-consultation record template. The conversion of the second keyword may be based on a term conversion model or on a knowledge dictionary. In some embodiments, the second inquiry may be executed by a terminal device other than the patient terminal device (e.g., a waiting terminal device). In some embodiments, process 900 may be executed by an agent corresponding to a medical outpatient service or a medical consultation process. For example, a pre-consultation service may be provided by a pre-consultation agent.

[0216] Fig.10 is a flowchart of an exemplary process 1000 for providing medical consultation services based on sensory information according to some embodiments of the present specification. In some embodiments, Fig.10 The process 1000 shown in FIG. 6 can be executed in a target medical scenario, which involves a medical consultation service provided to a patient in a consulting room in the consultation session 630. The process 1000 may include one or more of subprocesses 1010, 1020, 1030, 1040, and 1050. In some embodiments, at least a portion of the process 1000 may be executed by a consultation agent corresponding to the first terminal device.

[0217] Subprocess 1010 may be used to display interface elements related to the patient's pre-consultation record based on the sensory information collected by one or more sensory devices in the consulting room. Subprocess 1010 may be executed in the consultation session 630. Fig.10 As shown, subprocess 1010 may include steps 1012 and 1014 .

[0218] Step 1012, determining whether the patient has started the consultation based on the sensed information.

[0219] For example, the processing device 210 may determine that the patient has started the consultation based on the sensory information (eg, images) of the patient entering the consulting room collected by a sensory device (eg, an image sensor).

[0220] Step 1014: In response to determining that the patient has started the consultation, the first terminal device may display an interface element (eg, third interface element 1130) related to the patient's pre-consultation record. For more information about the pre-consultation record, see Figure 8 Related instructions.

[0221] Subprocess 1020 may be used to generate a target diagnosis record based on the perception information. Subprocess 1020 may be executed at the end of the consultation session 630. Fig.10 As shown, subprocess 1020 may include steps 1022 , 1024 , and 1026 .

[0222] In step 1022, an initial diagnosis record may be generated based on the perception information.

[0223] The initial diagnosis record may be an automatically generated diagnosis record. In some embodiments, the initial diagnosis record may include an initial patient medical record, an initial diagnosis opinion, an initial diagnosis prescription (e.g., an initial treatment prescription and an initial examination prescription), an initial doctor's order, etc. In some embodiments, key content may be extracted from the perceived information based on a diagnosis record template. Key content refers to content related to a template field in a diagnosis record template. Key content may be converted into professional content based on a knowledge dictionary or a term conversion model. For example, with key content as an index, the corresponding professional language is retrieved from a knowledge dictionary as professional content. Further, an initial diagnosis record is generated by updating the diagnosis record template based on professional content and a knowledge database. The knowledge database refers to a knowledge database of a registration department, for example, including the department's consultation specifications (e.g., disease description specifications, diagnosis specifications, prescription specifications, doctor's order specifications, etc.).

[0224] In some embodiments, the patient's physical examination data collected by one or more examination devices during the consultation process can be obtained, and an initial diagnosis record can be further generated based on the physical examination data. In some embodiments, the initial diagnosis record can be generated by a consultation agent. The consultation agent can learn the mechanism of generating diagnosis records from various data (e.g., diagnosis record templates, knowledge dictionaries, knowledge databases, etc.), and process the perception information and patient data according to the learned mechanism to generate a diagnosis record.

[0225] Step 1024, present the initial diagnosis record to the doctor.

[0226] For example, the processing device 210 may control the first terminal device to present the initial diagnosis record. As another example, the processing device 210 may control the doctor terminal device to present the initial diagnosis record to the doctor. In some embodiments, the doctor terminal device may present the initial diagnosis record to the doctor at a preset time (such as after the doctor finishes the consultation work for the day). Alternatively, after the doctor enters a document review request, the doctor terminal device may present the initial diagnosis record to the doctor.

[0227] Step 1026, generating a target diagnosis record based on the initial diagnosis record and the feedback information input by the doctor on the initial diagnosis record.

[0228] The feedback information input by the doctor may include the doctor's modification and / or confirmation of the initial diagnosis record. For example, the doctor may input feedback information related to the initial diagnosis record through the first terminal device or the doctor's terminal device. The target diagnosis record is the diagnosis record modified and / or confirmed by the doctor. In some embodiments, the target diagnosis record may include the target patient's medical history, target diagnosis opinion, target diagnosis prescription (e.g., target treatment prescription and target examination prescription), target doctor's order, etc.

[0229] By generating target diagnosis records, manual writing errors in target diagnosis records can be reduced, and the efficiency of generating target diagnosis records can be improved. On the other hand, it can reduce the clerical work of doctors, allowing doctors to have more energy to care for patients and improve the quality of medical outpatient services.

[0230] Subprocess 1030 can be used to provide consultation advice based on the perceived information. Subprocess 1030 can be executed in the consultation session 630. Fig.10 As shown, subprocess 1030 may include operations 1032 and 1034 .

[0231] Step 1032, generating a consultation suggestion based on the sensed information and the patient data of the patient. Consultation suggestions refer to suggestions to assist doctors in providing medical consultation services. Exemplarily, consultation suggestions include supplementary inquiry suggestions, examination suggestions, prescription suggestions, treatment plan suggestions, etc.

[0232] In some embodiments, the consultation recommendation can be determined based on the knowledge database, consultation specifications, etc. corresponding to the registration department. For example, the processing device 210 can determine the conversation content between the doctor and the patient based on the perception information (for example, the voice signal collected by the sound sensor), and search in the knowledge database, consultation specifications, etc. based on the conversation content and / or patient data to determine the consultation recommendation. Exemplarily, the processing device 210 can search in the knowledge database and / or consultation specifications corresponding to the patient's registration department based on the conversation content and the patient's patient data to determine the information that has not been collected, and determine the supplementary inquiry suggestion based on the information that has not been collected. In some embodiments, the consultation recommendation can be generated based on the diagnostic model. For example, the processing device 210 can determine the model input based on the perception information and the patient data, input the model input into the diagnostic model, and the diagnostic model outputs the consultation recommendation corresponding to the model input.

[0233] In some embodiments, the consultation recommendations may be generated by a consultation agent. The consultation agent may learn the mechanism of generating consultation recommendations from various data (such as historical consultation records, knowledge databases, and consultation specifications), and process the perception information and patient data based on the learned mechanism to provide consultation recommendations.

[0234] Step 1034, control the first terminal device or the doctor terminal device to present the consultation suggestion. The consultation suggestion can improve the accuracy of diagnosis and prescription, and improve the efficiency of medical consultation services. As the consultation process progresses, the perception information will be updated in real time, and the consultation suggestion will be updated accordingly. For example, before the doctor examines the patient, the consultation suggestion may include a physical examination suggestion. When the doctor completes the physical examination of the patient, the consultation suggestion can be updated to a prescription suggestion.

[0235] Sub-process 1040 can be used to switch the display content of the first terminal device based on the perception information. Sub-process 1040 can be executed in the consultation link 630. Fig.10 As shown, subprocess 1040 may include operations 1042 and 1044 .

[0236] Step 1042, determining one or more keywords in the conversation between the doctor and the patient based on the perception information.

[0237] The keyword is related to the consultation service. For example, one or more keywords include keywords summarizing the condition and / or symptoms, such as disease site, symptom, time, intensity, frequency, etc. For another example, the keyword includes a command word.

[0238] Step 1044: Switch the display content of the first terminal device based on one or more keywords.

[0239] For example, the processing device 210 may switch the current display content of the first terminal device with the display content corresponding to the one or more keywords. For example, when the one or more keywords include physical examination data, the processing device 210 may display the physical examination data of the patient collected by the examination device to replace the current display content of the first terminal device.

[0240] Subprocess 1050 can be used to provide remote medical accompaniment services based on the perception information. The patient can request the remote medical accompaniment service in the pre-consultation. Subprocess 1050 can be executed in the consultation link 630. Fig.10 As shown, subprocess 1050 may include steps 1052 and 1054 .

[0241] Step 1052, determining whether the patient needs to communicate with the remote patient companion based on the perception information.

[0242] In some embodiments, the processing device 210 can detect whether the patient has issued a request to communicate with the remote accompanying person based on the perception information (e.g., voice data and / or image data). In some embodiments, the processing device 210 can determine the patient's state information based on the perception information, and determine whether the patient needs to communicate with the remote accompanying person based on the patient's state information. For example, when the state information indicates that the patient is in a state of high tension, fear, etc., the processing device 210 can determine that the patient needs to communicate with the remote accompanying person.

[0243] When it is determined that the patient needs to communicate with the remote consultant, the processing device 210 may execute step 1054 .

[0244] Step 1054 , the first terminal device may be controlled to enlarge the interface element related to the remote medical companion service (also referred to as the second interface element).

[0245] By enlarging the second interface element, the patient can view the real-time image of the remote patient and better communicate with the remote patient. In some embodiments, when the patient receives on-site medical outpatient services in the clinic, when it is detected that the patient needs to communicate with the remote patient, the processing device 210 can remind the patient to wear the XR device and control the XR device to present the image data of the remote patient.

[0246] In some embodiments of the present specification, the patient's communication needs can be detected based on the perception information, and the communication needs can be met in a timely manner, thereby providing the patient with more humane care and a more realistic and immersive medical companionship experience.

[0247] In some embodiments, subprocess 1040 may be used to further present medical data to a target user based on the perception information. For example, a control instruction issued by at least one target user may be obtained based on the perception information to retrieve at least a portion of the medical data. In response to the control instruction, at least the portion of the medical data may be retrieved and presented through the first terminal device.

[0248] Fig.12 1 is a schematic diagram of an exemplary hospitalization process 1200 according to some embodiments of the present specification. Fig.12 As shown, the hospitalization process 1200 includes an admission link 1210, an admission inquiry link 1220, a hospitalization link 1230, a discharge link 1240, a follow-up link 1250, etc., or any combination thereof. When the patient is in different links of the hospitalization process 1200, different user services may be provided to users related to the hospitalization process 1200. Users related to the hospitalization process may include the patient, medical service providers (e.g., doctors, nurses, etc.) who provide medical services to the patient in the hospitalization process, and visitors of the patient.

[0249] In some embodiments, the processing device 210 (eg, the service module 420 , an inpatient agent configured on the processing device 210 ) may execute the steps involved in multiple links of the inpatient process 1200 .

[0250] In the hospitalization admission link 1210, the patient can go through the relevant procedures for hospitalization. In some embodiments, hospitalization services related to the hospitalization admission link 1210 can be provided to relevant users. As an example only, in response to a hospitalization guidance request, the processing device 210 can obtain the first location of the patient's patient terminal device and the second location of the ward, and determine the planned path from the first location to the second location based on the real-time map of the hospital. In addition, the processing device 210 can instruct the patient terminal device to display guidance information related to the planned path to the patient. For example, the processing device 210 can instruct the patient terminal device to superimpose the guidance information on the patient's real-world view through augmented reality (AR) technology. For another example, the processing device 210 can instruct the patient terminal device to explain the hospitalization rules. For another example, the processing device 210 can determine whether the patient meets the conditions for admission examination in the ward. For more information about hospitalization services, see Fig.14 Related instructions.

[0251] In the admission inquiry link 1220, the patient may receive an admission inquiry, which is used to collect basic information related to the patient. In some embodiments, an admission inquiry service related to the admission inquiry link 1220 may be provided to relevant users. For example, the admission inquiry service may include performing one or more inquiries on the patient, generating an admission record for the patient, etc., or any combination thereof. For more information about the admission inquiry service, see Fig.15 Inquiries about the patient's admission can be made in the same way as Fig. 9 and Fig.10 The second inquiry described in is performed in a similar manner and will not be repeated here.

[0252] In the hospitalization phase 1230, the patient may stay in a hospital (e.g., a ward) for a period of time to receive round-the-clock medical care. In some embodiments, ward services related to the hospitalization phase 1230 may be provided to relevant users. For example, the ward services may include nursing services 1232, ward rounds services 1234, visit services 1236, etc., or any combination thereof.

[0253] Nursing services 1232 provide direct nursing care to patients, including administering medications, performing physical examinations, monitoring vital signs, and assisting with activities of daily living. For more information about nursing services, see Fig.16 and its related description.

[0254] The ward rounds service 1234 involves ward rounds conducted by the patient's medical team (e.g., at least one physician) in the ward, wherein the medical team can review and discuss the patient's status and care plan during the ward rounds. For example, the ward rounds service 1234 may include presenting data to facilitate communication between the physician and the patient, generating ward rounds notes, presenting a virtual ward space to one or more remote physicians, etc., or any combination thereof. For more information about the ward rounds service, see Fig.17 Related instructions.

[0255] The visit service 1236 allows remote visitors to communicate with patients remotely. For example, the visit service 1236 may include generating a virtual visit space for patients and remote visitors, presenting a virtual visit space to patients and remote visitors, etc., or any combination thereof.

[0256] As an example only, in response to a visit request, the processing device 210 may obtain the first current information of the patient and the second current information of the remote visitor, and generate a virtual visit space for the patient and the remote visitor based on the first current information and the second current information. The first current information may indicate the current state and / or current environment of the patient. The second current information may indicate the current state and / or current environment of the remote visitor. The virtual visit space refers to the digital environment presented to the patient and the remote visitor during the visit. In addition, the processing device 210 may present the virtual visit space to the patient and the remote visitor respectively through the public terminal device in the ward and the remote terminal device of the remote visitor.

[0257] In the discharge link 1240, the patient can go through the discharge procedures. In some embodiments, a discharge service related to the discharge link 1240 can be provided to the relevant user to guide the patient to be discharged from the hospital. As an example only, in response to a discharge instruction received from a doctor terminal device of a patient's doctor, the processing device 210 can obtain the patient's target hospitalization record. The target hospitalization record can record information about the patient's hospitalization process, such as medical history, treatment received, prescribed drugs, test results, and discharge summary. In addition, the processing device 210 can generate discharge data based on the target hospitalization record and present the discharge data to the patient through a public terminal device in the ward. The discharge data may include a discharge summary, a doctor's medical advice on discharge, guidance information on discharge procedures, guidance information on discharge process, discharge fees, payment methods, etc., or any combination thereof. Corresponding to determining that the patient performs a discharge operation, the processing device 210 can generate a discharge record corresponding to the patient. Discharge records are used to record relevant data of discharge, including discharge time, discharge summary, doctor's discharge instructions, discharge expenses, payment method, patient's status at discharge, etc.

[0258] In the follow-up link 1250, continuing care can be provided to the patient after he / she is discharged from the hospital to ensure continued recovery, address any remaining health issues, and prevent re-admission. In some embodiments, follow-up services related to the follow-up link 1250 can be provided to relevant users. As an example only, the processing device 210 can determine a follow-up plan for the patient based on the patient's target hospitalization record. The follow-up plan is used to guide how to provide follow-up services to the patient. In some embodiments, the follow-up plan may include one or more follow-ups performed at one or more planned times. In addition, the processing device 210 can enable the doctor's doctor terminal device and the patient's patient terminal device to remind the doctor and the patient respectively according to the follow-up plan. Follow-up can be performed offline or remotely in a virtual follow-up space.

[0259] In some embodiments, after the follow-up visit, the processing device 210 may generate a follow-up record corresponding to the patient. The follow-up record is used to record data related to the follow-up visit, including the time corresponding to the follow-up visit, the updated doctor's orders corresponding to the follow-up visit, the health monitoring information corresponding to the follow-up visit, etc. In some embodiments, the processing device 210 may update the follow-up plan. For example, the processing device 210 may obtain the patient's health monitoring information and update the follow-up plan based on the health monitoring information. The health monitoring information may be collected by one or more home monitoring devices in the patient's home.

[0260] Fig.13 1 is a schematic diagram of an exemplary process 1300 for providing inpatient services according to some embodiments of the present specification. At least a portion of the process 1300 may be performed by an inpatient agent.

[0261] In step 1302 , the processing device 210 may monitor a data source that collects data related to the patient's hospitalization process.

[0262] Data related to the hospitalization process can reflect the progress of the hospitalization process, the patient's status during the hospitalization process, and / or various aspects of the hospitalization process. Data source refers to the source (such as hardware equipment) that provides (such as collects) data related to the hospitalization process. For more information about hardware equipment, see Figure 1 Related instructions.

[0263] In some embodiments, the processing device 210 may monitor the data sources to detect data updates of at least one of the data sources. When at least one of the data sources collects updated data 1304 that has not been processed by the processing device 210, the data update may be detected. Updated data refers to data collected by at least one data source and not yet processed by the processing device 210. In some embodiments, the processing device 210 may be directly connected to the at least one data source in communication and directly monitor the at least one data source. Alternatively, the processing device 210 may establish a communication connection with a storage device (such as the storage device 230) storing patient data related to the patient collected by at least one data source, and monitor the at least one data source by monitoring the storage device. In some embodiments, the hospitalization process may include multiple stages. When the patient is in different stages of the hospitalization process, the processing device 210 may monitor different data sources. For example, the processing device 210 may determine the current stage of the patient in the hospitalization process and monitor at least a portion of the data source corresponding to the current stage. By way of example only, when the patient is in the admission stage, the processing device 210 may monitor the sensing device in the ward.

[0264] In response to detecting updated data 1304 in at least one of the data sources, the processing device 210 may perform EOI detection 1306 on the updated data 1304 collected by the at least one data source. EOI refers to a specific event or behavior that requires attention. EOI detection 1306 refers to processing the updated data collected by at least one data source to detect whether one or more EOIs have occurred. For example, exemplary EOIs may include: a patient is admitted to a ward, a preliminary examination of the patient is performed, at least one doctor is patrolling the ward, a nursing operation or medical examination is performed on the patient, the patient initiates a service request, obtains or updates the doctor's medical order for the patient, the patient's physiological state is abnormal, the doctor issues instructions to the patient, or the like, or any combination thereof.

[0265] In some embodiments, the at least one data source may include multiple data sources that collect data related to the same EOI. In some embodiments, the at least one data source may include a data source that collects data related to multiple EOIs.

[0266] In some embodiments, the processing device 210 may perform the EOI detection 1306 based on the EOI detection rules 1312. The EOI detection rules 1312 refer to rules that need to be engaged when performing the EOI detection 1306. In some embodiments, the EOI detection rules 1312 may be determined based on a history of EOI detections, or may be manually set by a user (e.g., a doctor, nurse, technician, etc.).

[0267] In some embodiments, each stage of the hospitalization process may correspond to one or more EOIs, and different stages of the hospitalization process may correspond to different types of EOIs. Therefore, the processing device 210 may perform EOI detection 1306 on the update data 1304 based on the current stage of the patient in the hospitalization process.

[0268] If EOI is detected 1308, the processing device 210 may perform one or more preset operations 1310 corresponding to the EOI to provide user services to the relevant users. Meanwhile, the processing device 210 may continue to perform operation 1302 and EOI detection 1306 to continuously monitor data updates and detect EOI.

[0269] In some embodiments, the processing device 210 may perform one or more preset operations 1310 corresponding to the EOI to provide hospitalization services to the relevant user.

[0270] Related users include any user associated with the inpatient procedure.

[0271] The preset operation 1310 may include general operations and / or specific operations. General operations refer to operations that need to be performed as long as an EOI occurs, regardless of the type of EOI. For example, general operations may include generating records related to events of interest based on updated data, sending records related to the EOI to relevant users for confirmation or sending them to a storage device for storage, etc. Specific operations refer to operations performed when a specific type of EOI occurs. For example, updating a patient's daily plan based on updated data can be determined as a specific operation corresponding to an EOI updated with a patient's medical order. Specific operations refer to operations performed when a specific type of EOI occurs. For example, updating a patient's daily plan based on updated data can be determined as a specific operation corresponding to an EOI that updates a doctor's medical order for a patient. As another example, sending a notification related to an event of interest to a medical service staff can be determined as a specific operation corresponding to an EOI of an abnormal physiological state of the patient.

[0272] For example, when the EOI includes a patient's admission to a ward, one or more preset operations 1310 may include: determining, based on the patient's patient data, the content of the inquiry to be conducted on the patient after the patient is admitted to the ward, so that the terminal device in the ward (such as the terminal device equipped on the smart bed) conducts inquiries based on the inquiry content; obtaining sensory information collected by one or more sensory devices in the ward during the inquiry, and generating a hospitalization record for the patient based on the sensory information.

[0273] As another example, when the EOI includes a request for obtaining hospitalization guidance, one or more preset operations 1310 may include obtaining a first location of a patient terminal device and a second location of a patient ward, determining a planned path from the first location to the second location based on a real-time map of the hospital, and instructing the patient's terminal device to present guidance information related to the planned path to the patient.

[0274] As another example, when the EOI includes satisfying a condition for performing a preliminary examination, one or more preset operations 1310 may include controlling the smart nursing cart to guide a nurse to a ward to perform a preliminary examination on the patient.

[0275] As another example, when the EOI includes obtaining or updating the patient's medical orders, the one or more preset operations 1310 may include determining the patient's daily plan based on the patient's patient data and the medical orders or updated medical orders for each day that the patient is hospitalized, presenting the daily plan to the patient through a terminal device in the patient's ward, and presenting the daily plan to the patient's corresponding nurse through the nurse's terminal device. The daily plan may include requiring at least one medical procedure to be performed on the patient each day.

[0276] As another example, when the EOI includes at least one doctor conducting ward rounds in a patient's ward, one or more preset operations 1310 may include obtaining sensory information collected by one or more sensory devices in the ward while the at least one doctor is conducting ward rounds, and generating a ward round record based on the sensory information.

[0277] As another example, when the EOI includes obtaining a visiting request, one or more preset operations 1310 may include obtaining first current information of the patient and second current information of the remote visitor, generating a virtual visiting space for the patient and the remote visitor based on the first current information and the second current information, and presenting the virtual visiting space to the patient and the remote visitor, respectively.

[0278] As another example, when the EOI includes a received discharge instruction, one or more preset operations 1310 may include obtaining the patient's target hospitalization record, generating discharge data based on the target hospitalization record, and presenting the discharge data to the patient via a terminal device in the patient's ward.

[0279] As another example, when the EOI includes that the patient has been discharged, the one or more preset operations 1310 may include determining a follow-up plan for the patient based on the patient's target hospitalization record. The follow-up plan may include one or more follow-ups performed at one or more planned times. For each of the one or more follow-ups, the one or more preset operations may also include causing the doctor's terminal device and the patient's terminal device to remind the doctor and the patient, respectively, according to the planned time of the follow-up.

[0280] In some embodiments, each stage of the hospitalization process may correspond to one or more events of interest, and different stages of the hospitalization process may correspond to different types of events of interest. Therefore, the processing device 210 may detect events of interest on the updated data according to the current stage of the patient in the hospitalization process. For example, the processing device 210 may determine the current stage of the patient in the hospitalization process, and determine the type data of the event of interest that needs to be detected according to the current stage of the patient. Further, the processing device 210 may detect events of interest on the updated data according to the type data.

[0281] Just as an example, when the patient is in the admission stage of the hospitalization process, the processing device 210 may determine that the events of interest that need to be detected include the patient's admission to the ward, the patient's satisfaction of the conditions for admission examination, the patient's admission examination, etc. For another example, when the patient is in the hospitalization stage, the processing device 210 may determine that the events of interest that need to be detected include at least one doctor's ward rounds in the ward, the patient's nursing operations or medical examinations, the patient's initiation of a service request, the patient's medical orders being obtained or updated, the patient's physiological condition being abnormal, and the doctor's instructions on the patient, etc.

[0282] By detecting events of interest on the updated data based on the patient's current stage in the hospitalization process, only specific types of events of interest need to be detected or tracked, which can effectively reduce the amount of data that needs to be processed, thereby reducing the processing requirements for the processing device 210 and improving the detection efficiency of events of interest.

[0283] In some embodiments, the processing device 210 may determine a correspondence 1314 between an EOI and a preset operation, and determine one or more preset operations 1310 corresponding to the detected EOI based on the correspondence 1314. The correspondence 1314 may reflect one or more preset operations 1310 that need to be performed when a specific type of EOI occurs. For example, the correspondence 1314 may be in the form of a lookup table.

[0284] In some embodiments of the present specification, a data source for collecting information related to the patient's hospitalization process is monitored. This monitoring helps to detect data updates and the occurrence of EOI in a timely manner, thereby quickly triggering corresponding preset operations. Therefore, hospitalization services can be automatically and efficiently provided to relevant users, thereby improving service efficiency and quality. In addition, the monitored data source can collect multimodal data at different stages of the entire hospitalization process, thereby achieving patient-centered healthcare and comprehensive hospital services.

[0285] In some embodiments, the processing device 210 may be configured with an agent (e.g., an inpatient agent corresponding to a smart bed, a nursing agent corresponding to a smart nursing cart, etc.), and the agent may execute at least a portion of the process 1300. For example, the agent may learn an EOI detection rule 1312 from historical records, and perform EOI detection 1306 according to the EOI detection rule 1312. For another example, the agent may learn a correspondence 1314 between an EOI and a preset operation from historical records, and determine one or more preset operations 1310 corresponding to the EOI according to the correspondence 1314.

[0286] In some embodiments, the agent may further learn EOI detection rules 1312 and / or the correspondence 1314 between EOI and preset operations based on patient data of different patients.

[0287] By integrating intelligent agents, the system can continuously learn EOI detection rules and / or corresponding relationships using big data technology, machine learning technology and other advanced methods. This continuous optimization of EOI detection rules and / or corresponding relationships can improve the accuracy, efficiency and service quality of inpatient services.

[0288] Fig.14 1400 is a schematic diagram of a process 1400 for providing hospitalization services according to some embodiments of the present specification. When the processing device 210 detects that the patient has been admitted to the hospital (for example, the patient has completed the relevant process of admission) or in a target medical scenario, the process 1400 can be executed. The target medical scenario is related to the hospitalization services provided to the patient in the ward. In some embodiments, at least a part of the process 1400 can be executed by a nursing agent corresponding to the smart nursing cart used in the ward.

[0289] like Fig.14 As shown, when the patient 261 is admitted to the ward, the processing device 210 (e.g., the service module 420) can determine whether the patient meets the conditions for admission examination after admission to the ward. For example, the processing device 210 can determine whether the patient has been in the ward for a certain time based on the image data, sound data, etc. collected by the sensing device in the ward to determine whether the patient meets the conditions for admission examination. In response to determining that the patient meets the conditions for admission examination, the processing device 210 can control the smart nursing cart 240-4 based on the real-time map of the hospital to guide the nurse to the ward and perform admission examination 1402 on the patient 261. After the nurse arrives at the ward, the smart nursing cart 240-4 can display information related to the admission examination to assist the nurse in conducting the admission examination.

[0290] In some embodiments, the processing device 210 can display the patient's hospitalization notice to the nurse through the smart nursing cart. The hospitalization notice may include at least one of the patient's patient data, hospital supplies list, ward information, and admission examination information.

[0291] After performing the admission examination 1402, the processing device 210 may obtain the first physical examination data 1404 related to the patient 261 collected by one or more examination devices during the admission examination 1402. The examination device may be an IoT device. Optionally, one or more examination devices may be accommodated in the smart nursing cart 240-4. In addition, the processing device 210 may generate a registration record 1406 of the patient 261 based on the first physical examination data 1404.

[0292] In some embodiments, the processing device 210 may display the registration record 1406 to the nurse through the smart care cart 240-4, and obtain feedback information 1408 about the registration record 1406 input by the nurse through the smart care cart 240-4. The feedback information 1408 may include a confirmation instruction, a modification instruction, etc. input by the nurse. Then, the processing device 210 may generate a target registration record 1410 based on the registration record 1406 and the feedback information 1408. For example, the processing device 210 may generate the target registration record 1410 by updating the registration record 1406 based on the feedback information 1408.

[0293] In some embodiments of the present specification, admission services (such as admission examinations) can be provided to patients in a semi-automated manner with the assistance of a smart nursing cart and / or a nursing agent corresponding to the smart nursing cart, which can reduce labor costs and improve the efficiency of admission services.

[0294] Fig.15 1 is a schematic diagram of an exemplary process 1500 for providing admission inquiry services according to some embodiments of this specification. In some embodiments, process 1500 can be performed in the admission inquiry stage or in a target medical scenario. The target medical scenario is related to the inpatient services provided to the patient in the ward. In some embodiments, process 1500 can be performed when the processing device 210 detects that the patient has arrived at the ward and / or the admission examination has been completed. In some embodiments, at least a portion of process 1500 can be performed by the inpatient agent of the smart bed and / or the nursing agent corresponding to the smart nursing cart.

[0295] In step 1502 , the processing device 210 (eg, the service module 420 ) may determine, based on the patient data of the patient, a third inquiry content to be conducted on the patient after the patient is admitted to the ward.

[0296] Patient data may include basic data (such as name, age, gender, weight, address, job, etc.), health data (such as disease type, disease symptoms), historical data (such as historical clinical data, historical hospitalization data, etc.), registration data (such as target registration records, etc.), etc. or any combination thereof. In some embodiments, the patient data includes the patient's electronic health record.

[0297] The third inquiry is also called the admission inquiry, which is used to collect basic information about the patient in order to make a preliminary judgment on the patient's condition and / or to preliminarily resolve the patient's confusion. The third inquiry may include questions that the patient needs to answer in the third inquiry.

[0298] In some embodiments, the processing device 210 may determine the third query content of the third query based on the patient data of the patient. The third query may be determined in a manner similar to the first query or the second query. For example, the processing device 210 may obtain a query template based on the patient data, determine missing information that has not been collected in the query template based on the query template and the patient data, and determine the third query content based on the missing information.

[0299] In some embodiments, the third inquiry includes multiple rounds of inquiry, each round of inquiry may include one inquiry and one answer from the patient. The third inquiry content may include the inquiry content of each round of inquiry. Alternatively, the third inquiry content may only include the inquiry content of the third round of inquiry.

[0300] In step 1504, the processing device 210 (eg, the service module 420) may enable a third terminal device in the ward to conduct a third inquiry based on the third inquiry content.

[0301] The third terminal device may include an XR device and / or a display device.

[0302] The third query can be similar to Figure 8 and Fig. 9 For example, based on the third inquiry content, a third inquiry is made to the patient through a third terminal device in the ward. The third terminal device may make the third inquiry by presenting a virtual character. The appearance characteristics of the virtual character may be determined based on the optical image data of the attending physician corresponding to the patient. In some embodiments, the processing device 210 may determine the acoustic characteristics of the virtual character based on the patient data, and make the third inquiry through the third terminal device based on the third inquiry content and the acoustic characteristics.

[0303] In step 1506, the processing device 210 (eg, the service module 420) may obtain sensory information collected by one or more sensory devices in the ward during the third inquiry.

[0304] The sensory information may include information related to the patient's answers during the third inquiry. The sensory information may include voice signals collected by one or more sound sensors, video signals collected by one or more image sensors, and the like.

[0305] In step 1508 , the processing device 210 (eg, the service module 420 ) may generate an admission record of the patient based on the sensing information.

[0306] The admission record is used to record detailed information of the patient's admission. In some embodiments, the admission record may include patient information (such as patient data), admission information, first inquiry information, etc. In some embodiments, the processing device 210 may generate the patient's admission record based on the sensory information and optional patient data. For example, the processing device 210 may obtain an admission template and generate an admission record based on the admission template, the sensory information, and the patient data.

[0307] In some embodiments, after generating the admission record, the processing device 210 may enable the doctor terminal device to submit the admission record to the doctor for confirmation. If the doctor confirms the admission record, the processing device 210 may transmit the admission record to the storage device for storage. If the doctor feedback needs to include more information in the admission record, the processing device 210 may update the admission record.

[0308] In some embodiments, the processing device 210 may obtain feedback information related to the admission record input by the doctor through the doctor's terminal device. The feedback information may include information that is not in the admission record but is deemed necessary by the doctor. Further, the processing device 210 may determine the supplementary inquiry content of the supplementary inquiry based on the feedback information, and enable the third terminal device in the ward to conduct a supplementary inquiry based on the supplementary inquiry content. The supplementary inquiry may be conducted in a manner similar to the admission inquiry. The processing device 210 may then obtain supplementary perception information collected by one or more perception devices during the supplementary inquiry, and update the admission record based on the supplementary perception information. Alternatively, the doctor may go directly to the ward to conduct a supplementary inquiry on the patient. The processing device 210 may obtain supplementary perception information collected by one or more perception devices during the supplementary inquiry. The processing device 210 may further update the admission record based on the supplementary perception information.

[0309] In some embodiments of the present specification, a third terminal device may be used to automatically perform admission inquiries and automatically generate admission records, which may reduce the workload of medical service providers.

[0310] Fig.161 is a schematic diagram of an exemplary process 1600 for providing nursing services according to some embodiments of the present specification. In some embodiments, the process 1600 can be performed every day during the patient's hospitalization to provide nursing services to the patient, or the process 1600 can be performed in a target medical scenario. The target medical scenario is related to the hospitalization services provided to the patient in the ward.

[0311] In step 1602 , the processing device 210 may determine a daily plan for the patient based on the patient data of the patient and the doctor's instructions to the patient.

[0312] The doctor's order to the patient refers to the doctor's instructions or instructions to the patient. In some embodiments, the patient's order can be stored in a storage device, and can be updated when any doctor issues a new order to the patient. The processing device 210 can obtain the latest version of the order from the storage device. In some embodiments, the processing device 210 can monitor various hardware devices to detect whether the doctor's order to the patient is updated. For example, the processing device 210 can monitor the data source for collecting relevant data of the patient's hospitalization process, and when it is detected that at least one data source has data update, it is determined whether the patient's order is updated based on the updated data obtained from the at least one data source. In response to detecting that the order is updated, the processing device 210 can update the daily plan based on the updated order to obtain an updated daily plan. As an example only, when providing admission inquiry service and / or ward rounds service to the patient, the patient's doctor may issue an updated order to the patient. The processing device 210 can detect the updated order based on the sensory information collected by the sensory device during the admission inquiry service and / or ward rounds service. Once the updated order is detected, the updated order can be stored in the storage device. As another example, the doctor may update the patient's medical orders stored in the storage device through the doctor's terminal device. In some embodiments, the processing device 210 may determine the doctor's medical orders based on the patient's electronic health record.

[0313] In some embodiments, the processing device 210 can determine the patient's daily plan based on the patient's patient data and the doctor's instructions (or updated instructions) to the patient. The daily plan may include at least one medical operation that needs to be performed on the patient that day. Exemplary medical operations may include nursing operations, inspection operations, etc. In some embodiments, the processing device 210 can obtain the patient's historical nursing plan and update the historical nursing plan based on the patient data and the doctor's instructions to determine the daily plan.

[0314] In some embodiments, the processing device 210 can update the patient's daily plan. For example, in response to receiving the patient's examination results, the processing device 210 can determine whether the daily plan needs to be updated based on the examination results; if it is determined that it is necessary, the daily plan is updated. For example, if the examination results indicate that the patient is getting better, the daily plan can be updated by reducing the dosage in the daily plan. For another example, in response to receiving a patient's request, the processing device 210 can update the daily plan based on the patient's request. For example, the processing device 210 can identify the request from the patient's voice signal and update the daily plan based on the identified patient request.

[0315] In some embodiments, the processing device 210 may mark the updated content in the updated daily plan. For example, the processing device 210 may mark the updated content in the daily plan by using different colors. For another example, the processing device 210 may mark the updated content in the daily plan by a dotted box. In some embodiments, the processing device 210 may send the daily plan (or the updated daily plan, the daily plan marked with the updated content) to the second terminal device of the patient's doctor (e.g., the attending physician) for confirmation.

[0316] In step 1604, the processing device 210 may present the daily plan to the patient through a public terminal device (e.g., a third terminal device) in the ward. For example, the processing device 210 may present the daily plan to the patient in the form of text through the screen of the third terminal device. For another example, the processing device 210 may play the daily plan to the patient through the speaker of the third terminal device. For another example, the processing device 210 may display the first virtual character explaining the daily plan through the third terminal device.

[0317] In step 1606 , the processing device 210 may present the daily plan to the nurse corresponding to the patient through a nurse terminal device (eg, a terminal device in a nurse workstation, etc.).

[0318] Step 1608: When the daily plan includes at least one nursing operation, the nurse may perform at least one nursing operation on the patient, and the processing device 210 may assist the nurse in performing at least one nursing operation according to the daily plan.

[0319] like Fig.16As shown, for each of at least one nursing operation, the processing device 210 can control the intelligent nursing cart to guide the nurse to the ward to perform the nursing operation according to the scheduled time of the nursing operation. For example, before the scheduled time of the nursing operation, the intelligent nursing cart can be controlled to move to the nurse workstation to notify the nurse that a nursing operation is required for the patient. Then, the intelligent nursing cart can be controlled to move and guide the nurse to the patient's ward. The processing device 210 can further control the intelligent nursing cart to present nursing instructions for the nursing operation after the nurse arrives at the ward.

[0320] At step 1610 , the processing device 210 may generate a care record.

[0321] Nursing records refer to nursing operation records applied to patients and / or patient status (e.g., vital signs and other physiological measurements) before, after, or during nursing operations. In some embodiments, the processing device 210 can obtain sensory information collected by one or more sensory devices in the ward when performing at least one nursing operation, and generate nursing records based on the sensory information. In some embodiments, the nursing record can be displayed to the nurse for confirmation via a smart nursing cart or a nurse terminal device.

[0322] In some embodiments, the daily plan further includes monitoring the patient's physiological condition. For example, the processing device 210 can determine whether the patient's physiological condition is abnormal based on the sensory information collected in the ward. In response to determining that the patient's physiological condition is abnormal, the processing device 210 can control the smart nursing cart to guide the nurse to the patient's ward.

[0323] In some embodiments, the processing device 210 may be configured with an agent (e.g., a nursing agent corresponding to a smart nursing cart, an inpatient agent corresponding to a smart hospital bed), and the agent may participate in executing at least a portion of the process 1600. For example, the agent may determine a patient's daily plan, present the daily plan to the patient and / or nurse, assist the nurse in performing at least one nursing operation, and generate a nursing record. In some embodiments, the processing device 210 configured with the nursing agent may be integrated into a smart hospital bed, or integrated into a third terminal device in a ward, or integrated into a smart nursing cart.

[0324] In some embodiments of the present specification, the automatic generation of daily plans and nursing records can significantly reduce the workload of nurses. This automation allows nurses to focus more on direct patient care rather than administrative tasks. In addition, monitoring of doctor order updates ensures timely updates to daily plans. This proactive approach improves nursing efficiency and quality of care by ensuring that interventions and nursing plans are adjusted in a timely manner based on the latest medical instructions.

[0325] Fig.171700 is a schematic diagram of an exemplary process 1700 for providing ward rounds services according to some embodiments of the present specification. In some embodiments, process 1700 can be performed during the hospitalization stage. In some embodiments, process 1700 can be performed when processing device 210 detects that at least one doctor is making ward rounds in the patient's ward or in a target medical scenario. The target medical scenario is related to the hospitalization services provided to the patient in the ward. In some embodiments, at least a portion of process 1700 can be performed by an inpatient agent.

[0326] In 1702, the processing device 210 (e.g., the service module 420) may obtain the sensing information. When at least one doctor is making rounds in the ward, the sensing device in the ward may collect the sensing information. In some embodiments, the processing device 210 may detect the ward round (i.e., EOI) by analyzing the sensing information collected by the sensing device.

[0327] At 1706 , the processing device 210 (eg, the service module 420 ) may generate a ward round record.

[0328] The ward round record is used to record ward round related data, including time, participants, patient data, communication content between the patient and at least one doctor, medical orders issued during the ward round, etc. In some embodiments, the ward round record can be submitted to at least one doctor (or part thereof) for confirmation.

[0329] In some embodiments, the processing device 210 may generate a ward round record based on the sensory information and the second physical examination data. Fig.17 As shown, in 1704, the processing device 210 can further obtain the second physical examination data of the patient. The second physical examination data can be collected by one or more examination devices during the ward rounds. The second physical examination data is similar to Fig.14 Relevant first physical examination data.

[0330] In some embodiments, one or more remote doctors can participate in ward rounds remotely. For example, the processing device 210 can generate a virtual ward space 1710 based on the perception information, and display the virtual ward space 1710 to one or more remote doctors (e.g., doctor 1722, doctor 1724, and doctor 1726) through one or more XR devices of one or more remote doctors. A remote doctor refers to a doctor who is not in the ward. A virtual ward space refers to a digital environment used for ward rounds. For example, a virtual ward space can be a digital twin space of the patient's ward, which reflects the real-time situation of the ward and provides an immersive experience similar to being on the scene for remote doctors. Optionally, the remote doctor can communicate with at least one doctor and patient in the ward through an XR device. For example, the processing device 210 can obtain the voice data and image data of the remote doctor, and control the third terminal device in the ward to play the voice of the remote doctor based on the voice data, and control the third terminal device based on the image data to display a virtual character representing the remote doctor using extended reality technology.

[0331] In some embodiments, at least one doctor can interact with a third terminal device in the ward. For example, the processing device can analyze the perception information, and when the processing device 210 detects that the doctor issues an instruction to retrieve patient data (such as an electronic health record), the third terminal device can be controlled to display the patient data to at least one doctor and the patient to facilitate disease analysis. For another example, when the processing device 210 detects an instruction to display the second physical examination data, the third terminal device can be controlled to display the second physical examination data to at least one doctor and the patient.

[0332] In some embodiments, process 1700 may further include operation 1712. In 1712, before providing the ward round service, the processing device 210 may display the patient data of the patient to each of the at least one doctor (or a portion thereof) through the second terminal device of each of the at least one doctor (or a portion thereof). For example, the second terminal device may simultaneously display a virtual character representing the patient and the patient data (e.g., an electronic health record) of the patient within the doctor's field of view, wherein the virtual character may explain the patient's status to the doctor and provide the doctor with ward round suggestions. By showing the patient data to the doctor in advance, the doctor may better understand the patient's status, thereby improving the accuracy and efficiency of the ward round service.

[0333] In some embodiments, the processing device 210 may be configured with an agent (e.g., an inpatient agent, a ward round agent), and the agent may participate in the execution of process 1700. For example, the agent may control one or more sensing devices to collect fourth sensing information, control one or more examination devices to collect second physical examination data, provide a virtual ward space, and generate a ward round record. In some embodiments, the processing device 210 configured with a ward round agent may be integrated into a hospital bed, a third terminal device, or a patient care robot.

[0334] Fig.18 is an exemplary schematic diagram of a process 1800 for surgical planning and execution according to some embodiments of the present specification.

[0335] like Fig.18 As shown, the surgical procedure 1800 includes a surgical planning link 1810, a surgical simulation link 1820, a preoperative preparation link 1830, a surgical execution link 1840, a postoperative review link 1850, and other stages or any combination thereof. When the patient is at different stages of the surgical procedure 1800, different user services may be provided to users related to the surgical procedure 1800. The users related to the surgical procedure 1800 may include the patient, medical service providers (such as doctors, nurses, etc.) who provide medical services to the patient during the operation, and the patient's accompanying personnel.

[0336] In some embodiments, process 1800 may be performed by processing device 210 (e.g., service module 420, an agent corresponding to a medical service device used in a surgical service / process configured on processing device 210). For example, the medical service device includes an intelligent surgical terminal device, an operating room intelligent robot nurse, and the like.

[0337] Step 1810, perform surgical planning.

[0338] Surgical planning refers to the process of developing a surgical plan (e.g., an optimal surgical plan or multiple feasible surgical plans) for a patient. In some embodiments, the processing device 210 can generate a surgical plan based on patient data (e.g., patient personal data, historical diagnosis and treatment data, medical examination data, etc.), doctor feedback information, and / or perception information (e.g., second perception information). In some embodiments, the processing device 210 can generate an initial surgical plan based on patient data, present the patient data to the doctor through a doctor terminal device, and generate a surgical plan based on the initial surgical plan and the doctor's feedback information about the initial surgical plan input through the doctor terminal device.

[0339] In some embodiments, the processing device 210 can determine the difficulty coefficient of the operation based on the patient data, and further determine whether to hold an expert meeting based on the difficulty coefficient of the operation. In response to determining that an expert meeting is needed, the processing device 210 presents a virtual meeting space through the doctor's doctor terminal device (e.g., the second XR device) and the remote terminal device (e.g., the third XR device) of the remote expert. During the expert meeting, the processing device 210 obtains the second perception information collected by the doctor terminal device (e.g., the second XR device) and the remote terminal device (e.g., the third XR device). The processing device 210 generates a surgical plan based on the patient data and the second perception information.

[0340] In some embodiments, the processing device 210 can generate a risk assessment result of the surgical plan by processing the surgical plan and at least a portion of the patient data using the risk assessment model. The processing device 210 determines risk prevention measures based on the risk assessment result, and presents the risk assessment result and risk prevention measures of the surgical plan to the doctor.

[0341] In some embodiments, the processing device 210 can generate explanation materials for explaining the surgical plan. The explanation materials are used to explain information related to the surgical plan, such as the explanation of the surgical plan, the execution process of the surgical plan at the surgical site of the patient, the postoperative recovery process of the patient after using the surgical plan, etc. In some embodiments, the processing device 210 can simultaneously display the explanation materials to the patient and the doctor through at least one terminal device (for example, the patient's first XR device and the doctor's second XR device). Based on the perception information collected by the perception device during the explanation of the surgical plan, the processing device 210 can determine feedback information about the surgical plan, and confirm or update the surgical plan based on the feedback information. For example, the processing device 210 can determine a target surgical plan from multiple surgical plans based on the feedback information. The target surgical plan will be used to perform surgery on the patient's surgical site.

[0342] Step 1820, perform surgery simulation.

[0343] Surgical simulation refers to the process in which doctors practice surgery in a safe and controllable environment in order to improve surgical plans and / or improve their surgical skills. For example, for complex or rare surgeries, doctors can use XR devices to perform simulated surgery on virtual patients in a virtual surgical scene (e.g., an extended reality surgical scene) based on the surgical plan. Identify potential risk points during the operation and develop corresponding risk prevention measures. In addition, for multiple surgical plans, doctors can simulate each surgical plan in a virtual surgical scene through an extended reality device in order to compare the advantages and disadvantages of different surgical plans and determine the best surgical plan.

[0344] In some embodiments, the processing device 210 can generate a virtual surgical scene for surgical simulation according to the surgical plan, and present the virtual surgical scene to the doctor through the doctor's terminal device. The processing device 210 can obtain the interactive instructions related to the virtual surgical device input by the doctor through the doctor's terminal device or the interactive device corresponding to the virtual surgical device. The processing device 210 can update the virtual surgical site and virtual surgical device in the virtual surgical scene according to the interactive instructions.

[0345] In some embodiments, the processing device 210 may determine a possible emergency situation in the virtual surgical scene based on the interactive instruction. Based on the possible emergency situation, the processing device 210 may update the virtual surgical site and the virtual surgical equipment in the virtual surgical scene. As an example only, the processing device 210 may obtain simulation data of the virtual surgical site and the virtual surgical equipment in the surgical simulation process. The processing device 210 may determine whether the surgical plan needs to be optimized based on the simulation data. In response to determining that the surgical plan needs to be optimized, the processing device 210 may update the surgical plan based on the simulation data. In some embodiments, step 1820 may be omitted.

[0346] Step 1830, perform preoperative patient preparation.

[0347] Preoperative patient preparation refers to the preparations that a patient undergoes before a surgical procedure or before entering the operating room for a surgical procedure. Fig.18 As shown, preoperative patient preparation may include preoperative education 1832 (also referred to as preoperative care) and preoperative guidance 1834.

[0348] Preoperative education refers to the process of explaining the patient's condition and surgical plan to the patient and / or the patient's family, and inferring the patient's postoperative recovery status. For example, the processing device 210 can determine the patient's preoperative education materials based on the patient data and the surgical plan. In the process of transporting the patient to the waiting area, the processing device 210 can enable the patient's patient terminal device to provide preoperative education to the patient based on the preoperative education materials. As an example only, the processing device 210 can control the patient's first XR device to present a virtual nurse character based on the preoperative education materials during the process of transporting the patient to the waiting area, and provide preoperative education to the patient through the virtual nurse character.

[0349] For another example, the processing device 210 may generate explanatory materials for explaining the surgical plan, and display the explanatory materials to the patient and the doctor simultaneously through at least one terminal device. The explanatory materials may be generated based on the digital twin model of the surgical site of the patient. For example, the explanatory materials may include a surgical video showing the process of performing the surgery at the surgical site of the patient based on the surgical plan.

[0350] Preoperative guidance refers to the process of guiding the patient to complete preoperative preparations before surgery. Preoperative guidance may include patient reception, patient verification, preoperative cleaning, establishment of intravenous access, etc. For example, the processing device 210 uses the nurse agent to guide the nurse to perform preoperative cleaning and / or intravenous access establishment on the patient. The nurse agent can interact with the nurse through the terminal device (XR glasses or smart display terminal) worn by the nurse and guide the nurse. In some embodiments, the processing device 210 determines the planned route from the patient's current location to the waiting area of ​​the operating room. The processing device 210 can control the smart chair to transport the patient to the waiting area along the planned route. The processing device 210 can collect the patient's sensory information (e.g., biometric information) through one or more third sensory devices in the waiting area. The processing device 210 can verify the patient's identity based on the biometric information.

[0351] In some embodiments, during the process of transporting the patient to the waiting area, the processing device 210 can obtain third perception information related to a portion of the planned path from the current position of the smart chair to the waiting area (for example, a portion of the planned path that the smart chair has not passed through) through one or more fourth perception devices in the hospital. Based on the third perception information, the processing device 210 can determine the potential risks in the portion that the planned path has not passed through, and update the portion that has not passed through based on the potential risks. In some embodiments, the processing device 210 can determine the planned route from the patient's current position to the waiting area, and control the smart chair to transport the patient to the waiting area along the planned route.

[0352] In some embodiments, after the patient is transported to the waiting area, the processing device 210 can perform patient verification on the patient. Patient verification refers to checking whether the patient meets the conditions for performing the operation. The conditions for performing the operation include: whether the patient is the target patient who needs to perform the current operation (for example, verify whether the identity information (for example, number, name, gender, age, fingerprint, etc.) of the patient (current patient) matches the target patient who needs to perform the current operation), whether the operation that the patient needs to perform is the operation that is currently being scheduled (or to be performed), whether the patient's current physical condition (for example, vital signs such as heart rate, blood pressure, and respiratory rate) meets the requirements of the operation, whether the patient fasts and abstains from water according to the doctor's order, whether the patient does not carry items that affect the safety of the operation (for example, metal products, electronic products, etc.), and whether the patient's mental state is normal (for example, the patient does not feel anxious, the tension is low, etc.), etc. Among them, the mental state of the verification object can be determined based on the recognition of the expression of the verification object. It can be understood that in response to the verification object not meeting any one or more of the conditions for performing the operation, the operation on the patient is suspended or postponed. For example, the processing device 210 can verify the patient based on the sensing information collected by the sensing device in the waiting area. As an example only, the patient's identity can be verified based on the patient's physiological data (fingerprint data, voiceprint data, face data) collected by the sensing device.

[0353] In some embodiments, the processing device 210 may generate an explanation of the surgical plan based on the second perception information, and generate an explanation video of the surgical plan based on the surgical video and the explanation of the surgical plan.

[0354] In some embodiments, the instructional material may include the patient's recovery process, and the processing device 210 may predict the patient's recovery process after surgery based on the patient's patient data.

[0355] In some embodiments, the processing device 210 may obtain an interactive instruction related to the explanatory material from at least one terminal device, and update the explanatory material displayed by at least one terminal device based on the interactive instruction. The interactive instruction refers to an instruction for controlling or modifying the explanatory material input by a patient through a patient terminal device or by a doctor through a doctor terminal device.

[0356] In some embodiments, the processing device 210 may determine feedback information related to the surgical plan based on the second perception information collected by one or more second perception devices during the description of the surgical plan, and confirm or update the surgical plan for the patient based on the feedback information. In some embodiments, after determining the target surgical plan from multiple surgical plans, the processing device 210 may further obtain a first confirmation instruction and a second confirmation instruction. The first confirmation instruction is an instruction about the surgical plan input by the patient through the patient terminal device. The second confirmation instruction is an instruction about the surgical plan input by the patient's family through the home terminal device. In response to receiving the first confirmation instruction and the second confirmation instruction, the processing device 210 may cause the patient terminal device, the doctor terminal device, and the home terminal device to present the operation protocol respectively. The processing device 210 may obtain signature information of the surgical protocol from the patient terminal device, the doctor terminal device, and the home terminal device, respectively.

[0357] Preoperative guidance refers to the process of guiding patients to complete preoperative preparations before the operation.

[0358] In some embodiments, the processing device 210 may determine a planned path from the patient's current location to the waiting area, and control the smart wheelchair to transport the patient to the waiting area along the planned path. Fig.18 As shown, before performing a preoperative operation on the patient according to the surgical plan, the processing device 210 can determine a planned path from the current position 1803 (e.g., ward) of the patient 261 to the waiting area 1810. The processing device 210 can control the smart wheelchair 240-5 to transport the patient 261 from the ward 1803 to the waiting area 1810 along the planned path.

[0359] Step 1840, perform the surgery.

[0360] Performing surgery refers to the operation related to surgery performed on the patient after entering the operating room. In some embodiments, the processing device 210 can obtain the first sensing information collected by one or more sensing devices in the operating room during the patient's surgery, and perform EOI detection on the first sensing information. In response to determining that EOI occurs, the processing device 210 can perform one or more preset operations corresponding to EOI. For more information about the one or more preset operations, see Fig.19 Related instructions.

[0361] Step 1850, perform postoperative review.

[0362] Postoperative review may include at least one of updating a doctor's order report, generating surgical results and doctor's operation records (to facilitate review of the surgical process), and creating a postoperative care plan. In some embodiments, the processing device 210 may provide postoperative care for the patient. For example, the processing device 210 may monitor the patient's postoperative physical signs; update the doctor's order based on the patient's postoperative physical signs; determine a postoperative care plan based on the updated doctor's order; and control the smart care cart to provide care for the patient based on the postoperative care plan.

[0363] In some embodiments, the processing device 210 can monitor the patient's postoperative vital signs through vital sign monitoring instruments in the ward (e.g., electrocardiogram, blood pressure monitor, etc.) to determine whether the patient's postoperative vital signs are within the normal postoperative vital sign range, whether there are any abnormalities, or whether the recovery progress is normal.

[0364] The postoperative care plan refers to the care tasks that caregivers (e.g., nurses, care workers, etc.) need to perform during the patient's hospitalization after surgery. The postoperative care plan may include basic postoperative care, postoperative patient education, abnormality reporting, etc. Basic postoperative care includes incision management (e.g., dressing changes, etc.), diet management, rehabilitation training, etc. Postoperative patient education includes education on postoperative precautions and patient emotional comfort. Abnormality reporting includes reporting of abnormal vital signs of patients, reporting of abnormal emotions (e.g., patients are too irritable or pessimistic), reporting of incision infection, etc.

[0365] In some embodiments, the processing device 210 can update the postoperative care plan in real time according to the patient status during the care process.

[0366] Fig.19 1900 is a flowchart of an exemplary process 1900 of performing a surgery according to some embodiments of the present specification. In some embodiments, the process 1900 can be performed by the processing device 210 or an agent corresponding to a medical service device configured on the processing device 210, the medical service device is applied to a surgical service / process or applied to a target medical scenario, the target medical scenario is related to a surgical service provided to a patient in an operating room.

[0367] Step 1910, the intelligent robot nurse prepares surgical tools in the operating room based on the surgical plan.

[0368] In some embodiments, the processing device 210 can determine the type and quantity of surgical tools required based on the surgical plan, and control the intelligent robot nurse to prepare the surgical tools based on the type and quantity, and place the surgical tools at a predetermined position on the operating table. In some embodiments, operation 1910 can be performed by a robot nurse agent.

[0369] Step 1921, obtaining first perception information collected by one or more sensing devices in the operating room during the patient's surgery.

[0370] The sensing device refers to a sensing device in an operating room. For example, the sensing device may be a sound sensing device, a gesture recognition device, a motion recognition device, a force sensing device, an image capture device, etc. In some embodiments, the sensing device may be part of an operating room terminal device (e.g., a doctor's second terminal device). The first sensing information may include voice data, motion data, and / or image data. In some embodiments, the first sensing information includes an image of a surgical tool captured by an image sensor.

[0371] like Fig.19 As shown, in some embodiments, the processing device 210 may perform EOI detection on the first perception information (operation 1922). In response to detecting that an EOI occurs, one or more preset operations corresponding to the EOI are performed (operation 1923).

[0372] Step 1922, perform EOI detection on the first perception information.

[0373] EOI refers to an event that may occur during surgery and affect the surgical process or bring risks to the patient. For example, EOI may include a dangerous condition of the patient during surgery, improper operation of the surgical participant, insufficient number of surgical tools or consumables, etc. In some embodiments, EOI includes at least one of an instruction issued by a surgical participant, a detected surgical risk, an abnormal physiological state of the patient, the number of surgical tools being less than a threshold, and completion of surgery.

[0374] EOI detection refers to the process of determining whether EOI occurs based on the first perception information during the operation. EOI detection may include voice or gesture detection, image detection, physiological sign detection, etc. For example, the processing device 210 may perform voice recognition on the voice information in the first perception information to determine whether the voice information is an instruction issued by a surgical participant (e.g., an instruction issued by a surgical participant regarding a target surgical tool or consumable). In other embodiments, the processing device 210 may perform image recognition on the image of the surgical tool in the first perception information to determine the number of each type of surgical tool, and further determine whether the number of any type of surgical tool is less than a threshold.

[0375] Physiological sign detection refers to monitoring the patient's vital signs (such as heart rate, blood pressure, respiratory rate, etc.) to determine whether EOI has occurred during surgery. For example, the processing device 210 can monitor the patient's heart rate in real time during surgery and compare the heart rate with a normal heart rate range. In response to determining that the patient's heart rate is not within the normal range, the processing device 210 can consider that EOI has been detected and determine that EOI is an abnormal physiological state of the patient, for example, an abnormal heart rate.

[0376] In some embodiments, operation 1922 is performed by a surgical agent and / or a robotic nurse agent. In some embodiments, EOI detection is performed based on EOI detection rules learned by the surgical agent and / or the robotic nurse agent from historical records. In some embodiments, the surgical agent and the robotic nurse agent are configured to detect different types of EOIs. For example, the surgical agent is configured to detect the patient's status, the completion of the operation, etc.; the robotic nurse agent is configured to detect EOIs related to surgical tools and EOIs that need to be handled by an intelligent robotic nurse. In some embodiments, Fig.19 As shown, EOI detection can be further based on patient data. For example, different patients may have different EOI detection rules.

[0377] Step 1923: In response to detecting the occurrence of EOI, one or more preset operations corresponding to the EOI are executed.

[0378] A preset operation is a preset action to be taken based on the EOI. For example, if the EOI is severe bleeding in the patient, the preset operation can be a hemostatic measure (such as blood transfusion, suturing and ligation to stop bleeding, etc.).

[0379] In some embodiments, one or more preset operations corresponding to the EOI are determined based on the corresponding relationship between the EOI and the preset operations. The predetermined relationship is learned by the surgical agent and / or the robotic nurse agent from historical records. Fig.19 As shown, one or more preset operations corresponding to the EOI are further determined based on the patient data of the patient.

[0380] In some embodiments, executing one or more preset operations corresponding to EOI may include: after detecting that EOI occurs, the processing device 210 sends a reminder message to the surgical participant or controls the surgical equipment (such as the intelligent robotic nurse 240-7) to perform preset operations.

[0381] In some embodiments, the EOI includes an instruction of the target surgical tool issued by the surgical participant, and one or more preset operations corresponding to the EOI include controlling the intelligent mechanical nurse 240-7 to deliver the target surgical tool to the surgical participant. The target surgical tool refers to the surgical tool that the surgical participant currently wants to obtain. The instruction about the target surgical tool may include the type and / or specification of the target surgical tool. The method for identifying the instruction about the target surgical tool may be voice recognition, gesture recognition, etc. In some embodiments, the processing device 210 may send the above instruction to the intelligent mechanical nurse 240-7, and the intelligent mechanical nurse 240-7 may identify the target surgical tool from the operating table and deliver the target surgical tool to the surgical participant (e.g., surgeon). For example, the intelligent mechanical nurse 240-7 may identify the first position of the target surgical tool on the operating table based on an image of the surgical tool on the operating table, and identify the second position of the surgeon based on an image containing the surgeon, determine the moving route from the first position to the second position (including moving distance, moving angle, moving drop, etc.), and based on the moving route, transport the target surgical tool from the first position to the second position to pass it to the surgeon.

[0382] In some embodiments, the EOI includes that the number of surgical tools is less than a preset value, and one or more preset operations corresponding to the EOI include controlling the intelligent robot nurse 240-7 to replenish surgical tools. In some embodiments, for each type of surgical tool, the processing device 210 can determine whether the number of surgical tools is less than a preset value based on an image of the surgical tool. In some embodiments, the processing device 210 can predetermine preset values ​​corresponding to various surgical tools based on the surgical plan. In some embodiments, the processing device 210 can adjust the preset values ​​corresponding to various surgical tools in real time based on the surgical progress.

[0383] In some embodiments, the EOI includes a detected surgical risk, and one or more preset operations corresponding to the EOI include providing a reminder about the surgical risk. The reminder can be in the form of a voice reminder, a text reminder, etc. For example, a notification about the surgical risk can be provided to a doctor.

[0384] In some embodiments, the EOI includes the completion of a surgical procedure, and the one or more preset operations corresponding to the EOI include generating a surgical report based on the first perception information.

[0385] In some embodiments, Fig.19 As shown, the processing device 210 may generate a preliminary surgery report based on the first sensed information (operation 1930). The processing device 210 may display the preliminary surgery report to the doctor (operation 1940). The processing device 210 may generate a surgery report based on the preliminary surgery record and the doctor's feedback information on the preliminary surgery report.

[0386] The surgical report refers to the original recorded data of the information and events related to the surgery that occurred during the operation. For example, the preliminary surgical report may include surgery-related information, patient-related records, participant-related records, etc. In some embodiments, the processing device 210 may use a report generation model or a preset report template to generate a preliminary surgical report based on the first perception information. Feedback information refers to the modification or confirmation made by the doctor to the preliminary surgical record. The doctor can input feedback information in the form of text input or voice input. The surgical report refers to the surgical report obtained after the preliminary surgical record is updated based on the feedback information.

[0387] In some embodiments, the processing device 210 may also generate a preliminary medical order report at the same time, before or after generating the preliminary surgery report. In some embodiments, the processing device 210 may generate a preliminary medical order report based on the first perception information (eg, voice data of the surgeon).

[0388] The initial medical order report refers to the original postoperative medical order. The initial medical order report may include the normal range of postoperative vital signs, medication prescriptions, expected complications and corresponding measures, nursing requirements, etc. The normal range of postoperative vital signs, medication prescriptions (including drug types, frequency and total amount), and expected complications (such as fever, infection, etc.) can be determined based on patient data, surgical reports, or physician voice signals.

[0389] In some embodiments, the processing device 210 may display the preliminary medical order report to the doctor. In some embodiments, the processing device 210 may display the preliminary medical order report to the doctor through the doctor's terminal device. The processing device 210 may obtain feedback information related to the medical order report input by the doctor through a sensing device (e.g., a microphone on the doctor's terminal device). The feedback information related to the medical order report includes the doctor's modifications or confirmations made to the preliminary medical order report (e.g., confirmation that the preliminary medical order report is correct). The processing device 210 may further generate a medical order report based on the preliminary medical order report and the feedback information related to the medical order report.

[0390] In some embodiments, the processing device 210 can simultaneously display the preliminary doctor's order report and the preliminary operation report to the doctor. In this case, the processing device 210 can obtain the feedback information related to the preliminary operation report and the feedback information related to the preliminary doctor's order report input by the doctor.

[0391] In some embodiments, operations 1930-1950 may be performed by a surgical agent corresponding to the intelligent surgical terminal device or an intelligent robotic nurse corresponding to the intelligent robotic nurse.

[0392] Fig. 202000 is a schematic diagram of an exemplary process 2000 for preoperative preparation and performing surgery according to other embodiments of the present specification. In some embodiments, the process 2000 can be executed by the processing device 210 or an agent corresponding to a medical service device used in a surgical service / procedure in a target medical scenario related to the surgical service provided to a patient in an operating room configured on the processing device 210.

[0393] Step 2010, formulate a surgical plan for the patient.

[0394] In some embodiments, the processing device 210 may determine the difficulty coefficient of the operation based on the patient data. The processing device 210 may determine whether to hold an expert meeting based on the difficulty coefficient of the operation. In response to determining that an expert meeting needs to be held, the processing device 210 may control the doctor's terminal device and the terminal device of the remote expert to display the virtual meeting space respectively. The processing device 210 may acquire the perception information collected by the doctor's terminal device and the terminal device during the expert meeting. The processing device 210 may generate a surgical plan based on the patient data and the acquired perception information.

[0395] In some embodiments, the processing device 210 may generate a preliminary surgical plan based on the patient data. The processing device 210 may display the preliminary surgical plan to the doctor. The processing device 210 may generate a surgical plan based on the preliminary surgical plan and feedback information about the preliminary surgical plan input by the doctor through the doctor terminal device.

[0396] Step 2020, provide preoperative care to the patient based on the surgical plan.

[0397] In some embodiments, the processing device 210 can determine the patient's preoperative care materials based on the patient data and the surgical plan. The processing device 210 can provide preoperative care to the patient based on the preoperative care materials using the patient terminal device during the process of transporting the patient to the waiting area. In some embodiments, preoperative care includes preoperative comfort and preoperative education. Preoperative comfort refers to preoperative preparations to reduce the patient's negative emotions (e.g., anxiety, tension, fear, etc.) before surgery through verbal communication, video, music, etc. Preoperative education refers to preoperative preparations to help patients understand the surgical process.

[0398] Step 2030, based on the surgical plan, controls the intelligent robot nurse 240-7 to prepare surgical tools in the operating room before the surgical procedure.

[0399] In some embodiments, the processing device 210 determines the type and quantity of surgical tools required based on the surgical plan, controls the intelligent robotic nurse 240-7 to prepare surgical tools based on the type and quantity, and places the corresponding surgical tools at predetermined locations in the operating room.

[0400] Step 2040, based on the sensory information collected by one or more sensory devices in the operating room during the operation, control the intelligent robot nurse 240-7 to assist in the operation.

[0401] In some embodiments, the processing device 210 can control the intelligent robot nurse 240-7 to replenish surgical tools. In some embodiments, the processing device 210 can control the intelligent robot nurse 240-7 to deliver the target surgical tools to the surgical participants. In some embodiments, operations 2030 and 2040 can be performed by a robot nurse agent.

[0402] After the operation is performed (i.e., postoperative matters), the processing device 210 can also control the intelligent robot nurse 240-7 to organize the operating room. Organizing the operating room refers to the process of cleaning or cleaning surgical equipment and surgical tools. For example, organizing the operating room may include cleaning discarded surgical tools, counting the number of remaining surgical tools, and disinfecting surgical equipment and surgical tools.

[0403] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0404] At the same time, this specification uses specific words to describe the embodiments of this specification. For example, "one embodiment", "an embodiment", and / or "some embodiments" refer to a certain feature, structure or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "one embodiment" or "an embodiment" or "an alternative embodiment" mentioned twice or more in different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures or characteristics in one or more embodiments of this specification can be appropriately combined.

[0405] In addition, unless explicitly stated in the claims, the order of processing elements and sequences, the use of alphanumeric characters, or the use of other names in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some invention embodiments that are currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0406] Similarly, it should be noted that in order to simplify the description disclosed in this specification and thus help understand one or more embodiments of the invention, in the above description of the embodiments of this specification, multiple features are sometimes combined into one embodiment, figure or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this specification are more than the features mentioned in the claims. In fact, the features of the embodiments are less than all the features of the single embodiment disclosed above.

[0407] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values ​​can be as accurate as possible within the feasible range.

[0408] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this specification is hereby incorporated by reference in its entirety. Except for application history documents that are inconsistent with or conflicting with the content of this specification, documents that limit the broadest scope of the claims of this specification (currently or later attached to this specification) are also excluded. It should be noted that if the descriptions, definitions, and / or use of terms in the materials attached to this specification are inconsistent or conflicting with the content of this specification, the descriptions, definitions, and / or use of terms in this specification shall prevail.

[0409] Finally, it should be understood that the embodiments in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A system for providing medical services, characterized in that: The system includes a processing device, a sensing device and a pre-diagnosis terminal device, wherein: The sensing device and the pre-consultation terminal device are communicatively connected with the processing device and applied in a target medical scenario, wherein the target medical scenario involves a pre-consultation service provided to a patient; The sensing device is configured to collect sensing information related to the target medical scenario and send the sensing information to the processing device; and The processing device is configured to maintain a pre-consultation agent corresponding to the pre-consultation terminal device, and implement the pre-consultation service by enabling the pre-consultation agent to participate in processing the perception information, wherein, to implement the pre-consultation service, the pre-consultation agent is configured as follows: Based on the department of the doctor with whom the patient is registered, displaying a virtual character to conduct a pre-consultation inquiry to the patient through the pre-consultation terminal device; A pre-consultation record is generated based on the perception information collected by the perception device in the pre-consultation inquiry.

2. The system according to claim 1, characterized in that The step of performing a pre-consultation inquiry on the patient by displaying a virtual character based on the department of the doctor registered by the patient through the pre-consultation terminal device includes: Determine the content of the pre-consultation questions based on the department of the doctor the patient is seeing; and Based on the pre-consultation inquiry content, the pre-consultation terminal device displays the virtual character to conduct the pre-consultation inquiry to the patient.

3. The system according to claim 2, characterized in that The content of the pre-consultation inquiry is determined based on the department of the doctor with whom the patient is registered, including: Obtain the pre-consultation record template corresponding to the doctor's department and the known information of the patient; By comparing the pre-consultation record template with the known information of the patient, determining the missing information that has not been collected in the pre-consultation record template; The pre-diagnosis inquiry content is determined based on the missing information.

4. The system according to claim 2, characterized in that The content of the pre-consultation inquiry is determined based on the department of the doctor with whom the patient is registered, including: Using the missing information determination model to process the department where the doctor is located and the known information of the patient to determine the missing information to be collected from the patient; The missing information is processed using a first inquiry content determination model to determine the pre-diagnosis inquiry content, wherein the missing information determination model and the first inquiry content determination model are trained machine learning models.

5. The system according to claim 1, wherein: The pre-consultation terminal device includes an extended reality device, and the extended reality device displays the virtual character through extended reality technology.

6. The system according to claim 1, wherein: The appearance characteristics of the virtual character are determined based on the optical image of the doctor and / or the basic information of the patient.

7. The system according to claim 2, characterized in that The controlling the pre-diagnosis terminal device to conduct a pre-diagnosis inquiry on the patient includes: After the patient completes the registration, the estimated waiting time of the patient is determined based on the doctor's consultation record of the day and the patient's registration record; In response to the estimated waiting time being greater than a first preset time threshold and less than a second preset time threshold, the pre-consultation inquiry is initiated to the patient via the pre-consultation terminal device, and the second preset time threshold is greater than the first preset time threshold.

8. The system according to claim 2, characterized in that The pre-diagnosis inquiry includes multiple rounds of inquiry, and the pre-diagnosis inquiry content includes the inquiry content of each round of inquiry. The initiating the pre-diagnosis inquiry to the patient includes: For each round of pre-diagnosis questions except the first round, Determining semantic information and emotional information of the patient's historical answers based on data collected before the current round of questioning; Based on the semantic information and the emotional information, adjusting the inquiry content of the current round of inquiry; Based on the adjusted inquiry content of the current round of inquiry, the current round of inquiry is performed through the pre-inquiry terminal device.

9. The system according to claim 8, characterized in that The step of performing the current round of inquiry based on the adjusted inquiry content of the current round of inquiry through the pre-inquiry terminal device includes: Determining the voice characteristics of the current round of questioning based on the semantic information and the emotional information; Based on the sound features, the pre-consultation terminal device is controlled to conduct a current round of inquiries to the patient.

10. The system according to claim 8, characterized in that The adjusting the inquiry content of the current round of inquiry based on the semantic information and the emotional information includes: Acquiring physiological status information corresponding to the patient's inquiry in the current round, wherein the physiological status information is collected using a wearable device worn by the patient; Based on the semantic information, the emotional information and the physiological state information, the inquiry content of the current round of inquiry is adjusted.

11. The system according to claim 10, characterized in that The pre-diagnosis agent is further configured as follows: determining a feedback parameter based on at least a portion of the semantic information, the emotional information, and the physiological state information; The wearable device is controlled to apply feedback to the patient based on the feedback parameter, and the feedback includes at least one of force feedback and temperature feedback.

12. The method according to claim 1, characterized in that The pre-diagnosis inquiry includes multiple rounds of inquiry, and the pre-diagnosis inquiry of the patient includes: For each current round of queries except the first round of queries, Based on the inquiry content of historical inquiries, the patient's historical answers and the known information of the patient, using a second inquiry content determination model to determine the inquiry content corresponding to the current round of inquiries, where the second inquiry content determination model is a trained machine learning model; Based on the inquiry content corresponding to the current round of inquiry, the current round of inquiry is performed through the patient terminal.

Images