Medical service method and system

Abstract

The invention provides a medical service system and method. The system comprises a hardware device and a processing device. The hardware device is configured to collect data related to a physical entity of a physical hospital, and the physical entity at least comprises the hardware device of the physical hospital and a medical service process. The processing equipment is configured to map the data to a virtual hospital corresponding to a physical hospital; and providing user services to related users of the physical hospital through the user space application. The virtual hospital includes a digital twin corresponding to a physical entity of the physical hospital, at least a portion of the digital twin being updated during the data mapping process. The user space application is configured to provide a related user of the physical hospital with an access portal to interact with the virtual hospital, at least a portion of the user service being provided to the related user based on the interaction between the related user and the at least one digital twin.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to International Application No. PCT / CN2024 / 109056, filed on July 31, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present specification relates to the field of medical services, and in particular, to a medical service system and method based on XR technology, AI technology, and / or other pioneering technologies. BACKGROUND

[0003] Currently, when a patient needs to receive multiple medical services in a hospital, he / she needs to go through a complex and fragmented medical service process. This is both cumbersome and time-consuming for the patient, making it more complicated for the patient to understand the medical service process and identify the appropriate facilities. Therefore, the overall efficiency of medical services is reduced. At the same time, these complexities also pose significant challenges to hospital staff in providing medical services and managing related medical resources.

[0004] The rapid development of cutting-edge technologies, such as extended reality (XR) technologies (e.g., augmented reality (AR) technologies, virtual reality (VR) technologies, mixed reality (MR) technologies, etc.), artificial intelligence (AI) technologies, digital twin technologies, Internet of Things (IOT) technologies, blockchain technologies, spatial computing technologies, image rendering technologies, etc., has brought about transformative changes and substantial benefits in different systems. However, despite these advances, the full utilization of these innovative technologies in hospital systems has not yet been realized. Therefore, there is a need to develop medical service systems and methods that integrate these advanced technologies to improve service efficiency, service accuracy, and patient satisfaction (e.g., patient experience, especially immersive medical service experience). SUMMARY

[0005] One aspect of embodiments of the present specification provides a medical service system. The medical service system can include a hardware device and a processing device. The hardware device can be configured to collect data related to a physical hospital. The processing device is configured to map the data related to the physical hospital into a virtual hospital corresponding to the physical hospital, and provide user services to a related user of the physical hospital through a user space application, wherein the user space application is configured to provide the related user of the physical hospital with an access portal for interacting with the virtual hospital, and provide at least a part of the user services to the related user according to the interaction between the related user and the virtual hospital.

[0006] One aspect of embodiments of the present specification provides a medical service system. The medical service system can include a hardware device and a processing device. The hardware device can be configured to collect data related to a physical hospital. The processing device can be configured to maintain a digital smart object, and provide a user service to a relevant user of the physical hospital through a user space application, so that the digital smart object participates in processing the data related to the physical hospital.

[0007] One aspect of embodiments of the present specification provides a medical service system. The system includes a hardware device and a processing device. The hardware device is configured to collect data related to physical entities of a physical hospital, the physical entities including at least the hardware device and a medical service process of the physical hospital. The processing device is configured to map the data into a virtual hospital corresponding to the physical hospital, and provide a user service to a relevant user of the physical hospital through a user space application. The virtual hospital includes digital twins corresponding to the physical entities of the physical hospital, at least part of the digital twins being updated in the data mapping process. The user space application is configured to provide the relevant user of the physical hospital with an access portal for interacting with the virtual hospital, at least part of the user service being provided to the relevant user based on the interaction between the relevant user and at least one digital twin.

[0008] One aspect of embodiments of the present specification provides a medical service system. The system includes a hardware device and a processing device. The hardware device is configured to collect data related to physical entities of a physical hospital. The processing device is configured to map the data into a virtual hospital corresponding to the physical hospital, and provide a user service to a relevant user of the physical hospital through a user space application. The virtual hospital includes an agent, the agent being self-evolved based on the data and artificial intelligence (AI) technology. The user space application is configured to provide the relevant user of the physical hospital with an access portal for interacting with the agent, at least part of the user service being provided to the relevant user based on the interaction between the relevant user and the agent.

[0009] Additional features will be set forth in the description that follows, and in part will be apparent from the description, or can be learned by practice of the application as hereinafter described, or can be learned by practice of the application. The features of the application can be realized and attained by means of the instruments, combinations of means, and steps described in the description that follows, and can be realized and attained by means of the instruments, combinations of means, and steps described in the description that follows. BRIEF DESCRIPTION OF DRAWINGS

[0010] The application is further described in terms of exemplary embodiments. These exemplary embodiments are described in detail in the description that follows, in reference to the drawings. These embodiments are non-limiting exemplary embodiments, in which like numbers represent similar structures throughout the several views of the drawings, and in which:

[0011] Figure 1 is a block diagram of an example medical service system shown in accordance with some embodiments of the present application;

[0012] Figure 2 is a schematic diagram of an example medical service system shown in accordance with some embodiments of the present application;

[0013] Figure 3 is a schematic diagram of an example hospital support platform shown in accordance with some embodiments of the present application;

[0014] Figure 4 is a block diagram of an example processing device shown in accordance with some embodiments of the present application;

[0015] Figure 5 is a schematic diagram of an example flow of providing user services shown in accordance with some embodiments of the present application;

[0016] Figure 6 is a schematic diagram of an example flow of providing user services shown in accordance with some embodiments of the present application;

[0017] Figure 7 is a flow diagram of an example flow of providing user services shown in accordance with some embodiments of the present application;

[0018] Figure 8 is a schematic diagram of an example medical visit flow shown in accordance with some embodiments of the present application;

[0019] Figure 9 is a schematic diagram of an example flow of providing pre-visit services shown in accordance with some embodiments of the present application;

[0020] Figure 10 is a flow diagram of an example flow of conducting a second inquiry shown in accordance with some embodiments of the present application;

[0021] Figure 11 is a flow diagram of an example flow for providing medical outpatient services based on perception information shown in accordance with some embodiments of the present application;

[0022] Figure 12 is a schematic diagram of an example visit interface shown in accordance with some embodiments of the present application;

[0023] Figure 13 is a schematic diagram of an example inpatient flow shown in accordance with some embodiments of the present application;

[0024] Figure 14 is a flow diagram of an example flow for providing inpatient services shown in accordance with some embodiments of the present application;

[0025] Figure 15is a schematic diagram of an exemplary flow for providing care services, shown in accordance with some embodiments of the present application;

[0026] Figure 16 is an exemplary schematic diagram of a flow for surgery planning and execution, shown in accordance with some embodiments of the present application;

[0027] Figure 17 is a schematic diagram of an exemplary flow for pre-operative education, shown in accordance with some embodiments of the present application;

[0028] Figure 18 is a schematic diagram of exemplary pre-operative guidance, shown in accordance with some embodiments of the present application;

[0029] Figure 19 is a schematic diagram of an exemplary flow for surgery execution, shown in accordance with some embodiments of the present application;

[0030] Figure 20 is a schematic diagram of an exemplary medical service flow, shown in accordance with some embodiments of the present application;

[0031] Figure 21 is a schematic diagram of an exemplary flow for determining a target episode, shown in accordance with some embodiments of the present application;

[0032] Figure 22 is a schematic diagram of exemplary first pre-set operations, shown in accordance with some embodiments of the present application;

[0033] Figure 23 is a schematic diagram flow chart of an exemplary flow for assisting a physician, shown in accordance with some embodiments of the present application;

[0034] Figure 24 is a schematic diagram of an exemplary interactive interface, shown in accordance with some embodiments of the present application; and

[0035] Figure 25 is a schematic diagram of an exemplary system for hospital management, shown in accordance with some embodiments of the present application. DETAILED DESCRIPTION

[0036] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to one skilled in the art that the present teachings can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high-level, without detail, in order to avoid unnecessarily obscuring the aspects of the application. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of the description. Thus, the description is not intended to be limited to the

[0037] The terminology used in this description is for the purpose of describing particular example embodiments only and is not intended to be limiting. As used in this description, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this description, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0038] It will be understood that when a unit, engine, module, or block is referred to as being "on," "connected to," or "coupled to" another unit, engine, module, or block, it can be directly on, connected, or coupled to the other unit, engine, module, or block, or it can be indirectly on, connected to, or coupled to the other unit, engine, module, or block via intervening units, engines, modules, or blocks, unless the context clearly indicates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0039] The features and characteristics of the present description, as well as the related structural elements of the embodiments, and the operational and functional characteristics of the components, and the combination of components and economies of manufacture, will be more readily understood upon reference to the following description. All the drawings are part of the present description. However, it should be expressly understood that the drawings are included only by way of illustration and do not limit the scope of the present description. It should be understood that the drawings are not necessarily to scale.

[0040] Figure 1 is a block diagram of an example medical service system 100 in accordance with some embodiments of the present application.

[0041] The medical service system 100, which can also be referred to as a meta-hospital system, is constructed based on a variety of innovative technologies, including meta-universe technology, XR technology (e.g., AR technology, VR technology, MR technology, etc.), AI technology, digital twin technology, IOT technology, data flow technology (e.g., blockchain technology, data privacy computing technology), space computing technology, image rendering technology, etc.

[0042] As shown in Figure 1 , the medical service system 100 can include a physical hospital 110, a virtual hospital 130, a user space application 120, and a hospital support platform 140. In some embodiments, the hospital support platform 140 can map data related to the physical hospital 110 into the virtual hospital 130 corresponding to the physical hospital 110, and provide user services to relevant users of the physical hospital 110 through the user space application 120. For example, at least part of the user services can be provided to the relevant users based on the interaction between the relevant users and the virtual hospital 130.

[0043] The physical hospital 110 refers to a hospital that exists in the physical world and has a tangible property (e.g., has a measurable property such as mass, volume, shape, etc., and can be perceived by human senses or instruments). In this context, a health care facility that provides medical, surgical, and psychiatric care and treatment for people is collectively referred to as a hospital.

[0044] As shown in Figure 1 , the physical hospital 110 can include a plurality of physical entities. For example, the plurality of physical entities can include a department, a user, a hardware device, a user service, a public area, a medical service process, etc., or any combination thereof.

[0045] The department refers to a specialized unit or department that provides a specific type of medical care, treatment, and service. Each department can focus on a certain specific medical field and can be equipped with healthcare professionals with expertise in that field. For example, according to a medical service process corresponding to each department, the department can include an outpatient department, an inpatient department, a surgical department, a support department (e.g., a registration department, a pharmacy department), etc., or any combination thereof. As another example, according to a medical field corresponding to each department, the department can include internal medicine, surgery, specialized medicine, a child care department, etc., or any combination thereof.

[0046] Users can include any user associated with Physical Hospital 110 (or referred to as a user related to Physical Hospital 110). For example, users can include patients (or parts of patients, such as organs), companions, visitors to patients, hospital staff of Physical Hospital 110, suppliers of Physical Hospital 110, application developers of Physical Hospital 110, etc., or any combination thereof. Hospital staff of Physical Hospital 110 can include healthcare providers (e.g., doctors, nurses, technicians, etc.), hospital administrators, support staff, or similar personnel, or any combination thereof. Exemplary hospital administrators can include ward nursing administrators, clinical administrators, ward directors, hospital directors, hospital administrators, functional administrators, or similar personnel, or any combination thereof.

[0047] The hardware devices may include hardware devices located in the physical hospital 110 and / or hardware devices communicating with hardware devices in the physical hospital 110. Exemplary hardware devices may include terminal devices, medical service devices, sensing devices, infrastructure devices, etc., or any combination thereof.

[0048] Terminal devices may include terminal devices that interact with users of the healthcare service system 100. For example, terminal devices may include terminal devices that interact with patients (also called patient terminals), terminal devices that interact with doctors (also called doctor terminals), terminal equipment that interacts with nurses (also called nurse terminals), terminal devices that interact with remote visitors (also called remote terminal devices), or public terminals in the hospital (e.g., clinic terminals, bedside terminal devices, terminal devices in waiting areas, smart surgical terminals), etc., or any combination thereof. In this application, unless clearly obtained from the context or otherwise specified, terminal devices owned by the user and terminal devices provided to the user by the physical hospital 110 will be collectively referred to as user terminal devices or user-interacting terminal devices.

[0049] Terminal devices may include mobile terminals, XR devices, smart wearable devices, etc. Mobile terminals may include smartphones, personal digital assistants (PDAs), displays, gaming devices, navigation devices, handheld terminals (POS), tablets, etc., or any combination thereof.

[0050] XR devices can include devices that allow users to participate in extended reality experiences. For example, XR devices can include VR components, AR components, MR components, etc., or any combination thereof. In some embodiments, XR devices can include XR headsets, XR glasses, XR patches, stereo headphones, etc., or any combination thereof. For example, XR devices can include Google Glass. TM Oculus Rift TM GearVR TM AppleVisionproTM In particular, the XR device can include a display component on which virtual content can be presented and / or displayed. In some embodiments, the XR device can further include an input component. The input component can enable user interaction between a user and virtual content (e.g., a virtual surgical environment) displayed by the display component. For example, the input component can include touch sensors, microphones, image sensors, etc. configured to receive user input that can be provided to the XR device and used to control the virtual world by changing visual content presented on the display component. The input component can include a hand grip, a glove, a stylus, a console, etc.

[0051] The smart wearable device can include a smart bracelet, smart footwear, smart glasses, smart headgear, smart watch, smart clothing, smart backpack, smart accessory, etc., or any combination thereof. In some embodiments, the smart wearable device can acquire physiological data (e.g., heart rate, blood pressure, body temperature, etc.) of a user.

[0052] The medical service device can be configured to provide medical services to a patient. For example, the medical service device can include an examination device, a care device, a treatment device, etc., or any combination thereof.

[0053] The examination device can be configured to provide examination services to a patient, e.g., to acquire examination data of the patient. Exemplary examination data can include heart rate, respiration rate, body temperature, blood pressure, medical imaging data, test report of body fluid (e.g., blood test report), etc., or any combination thereof. Accordingly, the examination device can include a vital sign monitor (e.g., blood pressure monitor, blood glucose meter, heart rate meter, thermometer, digital stethoscope, etc.), a medical imaging device (e.g., computed tomography (CT) device, digital subtraction angiography (DSA) device, magnetic resonance (MR) device, etc.), a laboratory device (e.g., blood routine examination device, etc.), or any combination thereof.

[0054] The care device can be configured to provide care services to a patient and / or assist a medical service provider to provide care services. Exemplary care devices can include a hospital bed, a patient care robot, a smart care cart, a smart medicine box, a smart wheelchair, etc.

[0055] The treatment device can be configured to provide treatment services to a patient and / or assist a medical service provider to provide treatment services. Exemplary treatment devices can include a surgical device, a radiotherapy device, a physical therapy device, etc., or any combination thereof.

[0056] The perception devices can be configured to collect perception information related to the environment in which they are located. For example, the perception devices can include image sensors, sound sensors, etc. The image sensors can be configured to collect image data in the physical hospital 110, and the sound sensors can be configured to collect voice signals in the physical hospital 110. In some embodiments, the perception devices can be standalone devices or can be integrated into another device. For example, the sound sensors can be part of a medical service device.

[0057] The infrastructure devices can be configured to support data transmission, storage, and processing. For example, the infrastructure devices can include networks, data center facilities, computing devices, computing chips, storage devices, etc.

[0058] In some embodiments, at least a portion of the hardware devices of the physical hospital 110 are Internet of Things (IoT) devices. IoT devices refer to devices with sensors, processing capabilities, software, and other technologies that connect and exchange data with other devices and systems through the Internet or other communication networks. For example, one or more of the medical service devices and / or the perception devices of the physical hospital 110 are IoT devices and are configured to transmit the collected data to the hospital support platform 140 for storage and / or processing.

[0059] The user services can include any services provided by the hospital support platform 140 to users. For example, the user services include medical services provided to patients and / or accompanying persons, support services provided to staff of the physical hospital 110 and / or suppliers of the physical hospital 110, etc. In some embodiments, the user services can be provided to patients, doctors, and hospital administrators through the user space application 120, which will be described in more detail below.

[0060] The public area refers to a shared space accessible to users (or a portion of users) in the physical hospital 110, which can also be referred to as the environment of the physical hospital 110. For example, the public area can include reception areas (e.g., front desk), waiting areas, corridors, and hallways, etc., or any combination thereof.

[0061] The medical service process refers to a process of providing corresponding medical services to a patient. The medical service process usually includes several links and / or steps that a user needs to go through to obtain corresponding medical services. Exemplary medical service processes can include outpatient processes, inpatient processes, surgical processes, or similar processes, or any combination thereof. In some embodiments, the medical service processes can include medical service processes corresponding to different departments, different diseases, etc. In some embodiments, a pre-set data collection protocol can be set and the standard links involved in the medical service process and how to collect data related to the medical service process can be specified. More descriptions about the medical service process can be found elsewhere in this application. For example, see Figure 5 and related descriptions thereof.

[0062] The user space application 120 provides access to user services provided by the hospital support platform 140 to users. The user space application 120 can be an application, a plug-in, a website, an applet, or any other suitable form. For example, the user space application 120 is an application installed on a user terminal device, which includes a user interface for users to initiate requests and receive corresponding services.

[0063] In some embodiments, the user space application 120 can include different applications corresponding to different types of users. For example, the user space application 120 includes a patient space application corresponding to patients, a doctor space application corresponding to doctors, a manager space application corresponding to managers, and the like, or any combination thereof. The user services provided through the patient space application, the doctor space application, and the manager space application are also referred to as patient space services, doctor space services, and manager space services, respectively. Exemplary patient space services include a registration service, a path guidance service, a pre-consultation service, a remote consultation service, an inpatient service, a discharge service, and the like. Exemplary doctor space services include a scheduling service, a surgery planning service, a surgery simulation service, a patient management service, a remote ward round service, a remote outpatient service, and the like. Exemplary manager space services include a monitoring service, a medical service evaluation service, a device parameter setting service, a service parameter setting service, a resource scheduling service, and the like.

[0064] In some embodiments, the patient space application, the doctor space application, and the manager space application can be integrated into one user space application 120, and the user space application 120 can be configured to provide access for each type of user (e.g., a patient, a medical service provider, a manager, and the like). For example only, a particular user can have a corresponding account that can be used to log in to the user space application, view corresponding medical data, and obtain corresponding user services.

[0065] According to some embodiments of the present application, by providing user space applications for different types of users, each type of user can easily obtain various user services that he / she can need on his / her corresponding user space application. In addition, users currently often need to install various applications to obtain different user services, which leads to poor user experience and high development cost. Therefore, the user space application of the present application can improve user experience, improve service quality and efficiency, enhance service security, and reduce development or operation cost.

[0066] In some embodiments, the user space application 120 can be configured to provide access entry for relevant users of the physical hospital 110 to interact with the virtual hospital 130. For example, through the user space application 120, a user can input an instruction for retrieving digital content of the virtual hospital 130 (e.g., digital twin models of hardware devices, patient organs, public areas), view the digital content, and interact with the digital content. As another example, through the user space application 120, a user can communicate with a virtual character representing an agent. In some embodiments, a public terminal of the hospital can install the user space application, and can log in an administrator account of a department corresponding to the public terminal in the user space application. A user can accept user services through the user space application installed in the public terminal.

[0067] The virtual hospital 130 is a digital twin (i.e., a virtual representation or virtual copy) of the physical hospital 110 for simulating, analyzing, predicting, and optimizing the operational status of the physical hospital 110. For example, the virtual hospital 130 can be a real-time digital copy of the physical hospital 110.

[0068] In some embodiments, the virtual hospital 130 can be presented to a user using digital technology. For example, when a relevant user interacts with the virtual hospital 130, at least a portion of the virtual hospital 130 can be presented to the relevant user using XR technology. As an example only, at least a portion of the virtual hospital 130 can be superimposed on a real-world view of the relevant user using MR technology.

[0069] In some embodiments, the virtual hospital 130 can include digital twins of physical entities related to the physical hospital 110. A digital twin refers to a virtual representation (e.g., a virtual copy, a mapping body, a digital simulator) of a physical entity. The digital twin can reflect and predict the state, behavior, and performance of the physical entity in real time. For example, the virtual hospital 130 can include digital twins of at least a portion of medical services, departments, users, hardware devices, user services, public areas, medical service processes, etc. of the physical hospital 110. The digital twin of a physical entity can have various forms, including models, images, graphs, texts, numerical values, etc. For example, the digital twin can be a virtual hospital corresponding to the physical hospital, a virtual person (e.g., a virtual doctor, a virtual nurse, and a virtual patient) corresponding to a personnel entity (e.g., a doctor, a nurse, and a patient), a virtual device (e.g., a virtual imaging device and a virtual scalpel) corresponding to a medical service device (e.g., an imaging device and a scalpel), etc.

[0070] In some embodiments, the digital twins can include one or more first digital twins and / or one or more second digital twins. The state of each first digital twin can be updated based on updates of the corresponding physical entity state. For example, in the process of mapping data related to the physical hospital 110 to the virtual hospital 130, one or more first digital twins can be updated. One or more second digital twins can be updated by at least one of the user space applications 120, and the update of each second digital twin can result in an update of the state of the corresponding physical entity. In other words, when the corresponding physical entity changes its state, the first digital twin can be updated accordingly; when the second digital twin is updated, the state of the corresponding physical entity also changes accordingly. For example, the one or more first digital twins can include digital twins of public areas, medical services, users, hardware devices, etc., and the one or more second digital twins can include digital twins of hardware devices, user services, medical service processes, etc. It should be understood that the digital twin can be a first digital twin or a second digital twin.

[0071] According to some embodiments of the present application, by generating the virtual hospital 130 including digital twins of physical entities related to the physical hospital 110, the physical hospital 110 (including hardware devices, users, user services, medical service processes, etc.) can be simulated and tested in a safe and controllable environment. Through virtual reality linkage (e.g., real-time interaction between the physical hospital 110 and the virtual hospital 130), various medical scenarios can be more accurately predicted and responded to, thereby improving the quality and efficiency of medical services. In addition, the use of XR technology and virtual reality integration technology makes the interaction of related users more natural, intuitive, and provides a more comfortable and efficient medical environment, thereby improving the user experience.

[0072] In some embodiments, the virtual hospital 130 can further include an intelligent agent that evolves itself based on data related to the physical hospital 110 and AI technology.

[0073] An agent refers to an agent that acts in an intelligent manner. For example, an agent can include a computing / software entity that can autonomously learn and evolve, and perceive and analyze data to perform a specific task and / or achieve a specific goal (e.g., a medical service process). Through AI technology (e.g., reinforcement learning, deep learning, etc.), an agent can continuously learn and self-optimize in interactions with an environment. In addition, an agent can collect and analyze massive data (e.g., relevant data of the physical hospital 110) through big data technology, and mine patterns and learn rules from the data to optimize decision-making processes, so as to identify changes in the environment, respond quickly, and make reasonable judgments in uncertain or dynamic environments. For example, an agent can autonomously learn and evolve based on AI technology to adapt to changes in the physical hospital 110. By way of example only, an agent can be built based on NLP technology (e.g., large language models, etc.), and can automatically learn and autonomously update through a large amount of language text (e.g., hospital business data and patient feedback information) to improve the quality of user services provided by the physical hospital 110.

[0074] In some embodiments, an agent can include different types of agents corresponding to different medical service processes, different user services, different departments, different diseases, different hospital positions (e.g., nurses, doctors, technicians, etc.), different links of a medical service process, etc. A specific type of agent is used to handle tasks corresponding to the specific type. In some embodiments, one agent can correspond to different medical service processes (or different medical services, or different departments, or different diseases, or different hospital positions). In some embodiments, an agent can operate in reference to basic configuration data (e.g., dictionaries, knowledge graphs, templates, etc.) of the department and / or disease corresponding to the agent. In some embodiments, multiple agents can communicate with each other and share information through a network to cooperatively complete complex tasks.

[0075] In some embodiments, the configuration of an agent can be set. For example, the basic configuration data used by an agent in operation can be set. The basic configuration data can include dictionaries, knowledge databases, templates, etc. For another example, the usage permissions of an agent can be set for different users. In some embodiments, the manager of the physical hospital 110 can set the configuration of an agent through the management space application.

[0076] In some embodiments, an agent can be integrated into or deployed on a hardware device. For example, an agent corresponding to inpatient services can be integrated into a hospital bed or a presentation device of a hospital bed. In some embodiments, an agent can be integrated into or deployed on an embodied intelligent robot. An embodied intelligent robot refers to a robotic system that combines physical presence (embodiment) with intelligent behavior (cognition). Embodied intelligent robots can be configured to interact with the real world in ways that mimic or complement human capabilities, utilizing physical form and cognitive functions to perform tasks, make decisions, and adapt to environments. By leveraging artificial intelligence and sensor technology, embodied intelligent robots can operate autonomously, interact with the environment, and continually improve performance. For example, an embodied intelligent robot can be configured with an agent corresponding to surgical services and assist a doctor in performing surgery.

[0077] In some embodiments, at least a portion of user services can be provided based on agents. For example, at least a portion of user services can be provided to a relevant user based on a processing result, where the processing result is generated by at least one of the agents based on data related to the physical hospital 110. As an example only, the data related to the physical hospital 110 can include data related to a medical service process of the physical hospital 110, the agents can include agents corresponding to the medical service process, and user services can be provided to a relevant user of the medical service process by processing the data using the agents corresponding to the medical service process.

[0078] The hospital support platform 140 can be configured to provide technical support to the medical service system 100. For example, the hospital support platform 140 can include computing hardware and software to support innovative technologies, including XR technologies, AI technologies, digital twin technologies, data flow technologies, etc. In some embodiments, the hospital support platform 140 can at least include a storage device for data storage and a processing device for data computation.

[0079] In some embodiments, the hospital support platform 140 can support interactions between the physical hospital 110 and the virtual hospital 130. For example, a processing device of the hospital support platform 140 can obtain data related to the physical hospital 110 from the hardware devices and map the data related to the physical hospital 110 into the virtual hospital 130. For example, the processing device of the hospital support platform 140 can update a portion of the digital twins (e.g., one or more first digital twins) in the virtual hospital 130 based on the obtained data, so that each portion of the digital twins in the virtual hospital 130 can reflect the updated status of the corresponding physical entity in the physical hospital 110. Based on such continuously updated digital twins corresponding to the respective physical entities, a user can have real-time understanding of the status of the physical entities related to the physical hospital 110, thereby achieving monitoring and evaluation of the physical entities. As another example, an agent corresponding to the data related to the physical hospital 110 can be trained and / or updated based on the data related to the physical hospital 110, thereby self-evolving and self-learning.

[0080] In some embodiments, the hospital support platform 140 can support and / or provide user services to relevant users of the physical hospital 110. For example, in response to receiving a user service request from a user, a processing device of the hospital support platform 140 can provide a user service corresponding to the service request. As another example, in response to detecting that a user service needs to be provided to a user, a processing device of the hospital support platform 140 can control a physical entity or a virtual entity corresponding to the user service to provide the user service. For example, in response to detecting that a patient is admitted to a hospital ward, a processing device of the hospital support platform 140 can control an intelligent nursing cart to guide a nurse to the hospital ward for admission examination of the patient.

[0081] In some embodiments, at least a portion of the user services can be provided to the relevant users based on interactions between the relevant users and the virtual hospital 130. The interactions refer to the interactions or influences (e.g., conversations, behaviors, etc.) between the relevant users and the virtual hospital 130. For example, the interactions between the relevant users and the virtual hospital 130 can include interactions between the relevant users and the digital twins in the virtual hospital 130, interactions between the relevant users and the agents, interactions between the relevant users and the virtual characters, etc., or any combination thereof.

[0082] In some embodiments, at least a portion of the user services can be provided to the relevant user based on the interaction between the relevant user and at least one of the digital twins. For example, an update instruction for a second digital twin input by the relevant user can be received through the user space application 120, and the corresponding physical entity of the second digital twin can be updated according to the update instruction. As another example, the user can view the first digital twin of the physical entity (e.g., a 3D digital twin model of a patient organ or a hardware device) through the user space application 120 to understand the status of the physical entity. Optionally, the user can change the display angle, display size, etc. of the digital twin. More descriptions about the digital twin of the physical entity can be found elsewhere in this application (e.g., in Figure 25 and related descriptions thereof).

[0083] In some embodiments, the processing device of the hospital support platform 140 can present a virtual character corresponding to the intelligent agent through the user space application, interact with the relevant user, and provide at least a portion of the user services to the relevant user based on the interaction between the relevant user and the virtual character.

[0084] In some embodiments, the hospital support platform 140 can have a five-layer structure, including a hardware device layer, an interface layer, a data processing layer, an application development layer, and a service layer. In some embodiments, the hardware devices of the physical hospital 110 can be part of the hospital support platform 140. More descriptions about the hospital support platform can be found elsewhere in this application (e.g., in Figure 3 and related descriptions thereof).

[0085] According to some embodiments of the present application, by comprehensively integrating various internal and external resources of the physical hospital (e.g., medical service devices, hospital personnel, drugs and consumables, etc.), a virtual hospital corresponding to the physical hospital can be established. The virtual hospital can reflect the real-time status (e.g., changes, updates, etc.) of the physical entities related to the physical hospital, thereby realizing the monitoring and evaluation of the physical entities. Such integration can provide accurate data support for the operation and intelligent decision-making of medical services. In addition, through the virtual hospital, users related to medical services can jointly establish an open and shared ecosystem, thereby promoting the innovation and improvement of medical services.

[0086] In addition, whole-life-cycle patient healthcare services can be provided within and outside the hospital. The perspective of medical services is expanded from pure disease treatment to cover the entire life cycle of patients, including prevention, diagnosis, treatment, rehabilitation, health management, etc. By establishing in-hospital and out-of-hospital linkage, the physical hospital can better integrate online and offline resources to provide comprehensive and continuous medical and health services for patients. For example, through remote monitoring and online consultation, the health status of patients can be followed up in real time, and treatment plans can be adjusted in a timely manner to improve treatment effects.

[0087] Figure 2 is a schematic diagram of an example healthcare system 200 according to some embodiments of the present application.

[0088] As shown in Figure 2 , the healthcare system 200 can include a processing device 210, a network 220, a storage device 230, one or more healthcare devices 240, one or more sensing devices 250, one or more patient terminals 260 of a patient 261, and one or more physician terminals 270 of a physician 271 associated with the patient 261. In some embodiments, the components in the healthcare system 200 can be connected and / or communicate with each other through wireless connections, wired connections, or a combination thereof. The connections among the components of the healthcare system 200 can be variable. For example only, the healthcare devices 240 can be connected to the processing device 210 through the network 220 or directly. As another example, the storage device 230 can be connected to the processing device 210 through the network 220 or directly.

[0089] The processing device 210 can process data and / or information obtained from the storage device 230, the healthcare devices 240, the sensing devices 250, the patient terminals 260, and / or the physician terminals 270. For example, the processing device 210 can map data related to a physical hospital to a virtual hospital corresponding to the physical hospital, and provide user services to the patient 261 and the physician 271 through the patient terminals 260 and / or the physician terminals 270, respectively, by processing the data related to the physical hospital. As another example, the processing device 210 can maintain a digital intelligent object, and provide user services to the patient 261 and the physician 271 through the patient terminals 260 and / or the physician terminals 270, respectively, by engaging the digital intelligent object in processing the data related to the physical hospital.

[0090] In some embodiments, the processing device 210 can be a single server or a group of servers. The group of servers can be centralized or distributed. In some embodiments, the processing device 210 can be located locally or remotely from the healthcare system 200. In some embodiments, the processing device 210 can be implemented on a cloud platform. For example, the cloud platform can include a private cloud, a public cloud, a hybrid cloud, a community cloud, a distributed cloud, an inter-cloud, a multi-cloud, etc., or a combination thereof. In some embodiments, the processing device 210 can be implemented through a computing device.

[0091] In some embodiments, the processing device 210 can include one or more processors (e.g., single-core processors or multi-core processors). For illustration only, only one processing device 210 is described in the healthcare system 200. However, it should be noted that the healthcare system 200 in the present application can also include multiple processing devices. Thus, operations and / or method steps performed by one processing device 210, as in the present application, can also be performed by multiple processing devices jointly or individually. For example, if in the present application, the processing device 210 of the healthcare system 200 performs procedure A and procedure B simultaneously, it should be understood that procedure A and procedure B can also be performed by two or more different processing devices in the healthcare system 200 jointly or individually (e.g., a first processing device performs procedure A, a second processing device performs procedure B; or the first and second processing devices jointly perform procedures A and B).

[0092] The network 220 can include any suitable network capable of facilitating the exchange of information and / or data of the healthcare system 200. The network 220 can be or include a wired network, a wireless network (e.g., an 802.11 network, a Wi-Fi network), a Bluetooth network, a near-field communication (NFC) network, and / or the like, or any combination thereof. TM The network 220 can include any suitable network capable of facilitating the exchange of information and / or data of the healthcare system 200. The network 220 can be or include a wired network, a wireless network (e.g., an 802.11 network, a Wi-Fi network), a Bluetooth network, a near-field communication (NFC) network, and / or the like, or any combination thereof.

[0093] The storage device 230 can store data, instructions, and / or any other information. In some embodiments, the storage device 230 can store data obtained from other components of the healthcare system 200. In some embodiments, the storage device 230 can store data and / or instructions that the processing device 210 can execute or use to perform the example methods described in the present application.

[0094] In some embodiments, the data stored in the storage device 230 can include multi-modal data. The multi-modal data can include multiple forms of data (e.g., images, graphs, videos, text, etc.), various types of data, data obtained from different sources, data related to different medical services (e.g., diagnosis, surgery, rehabilitation, etc.), data related to different users (e.g., patients, medical staff, administrators, etc.). For example, the data stored in the storage device 230 can include medical data of the patient 261 reflecting the health condition of the patient 261. For example, the medical data can include an electronic medical record of the patient 261. The electronic medical record refers to an electronic file recording various types of patient data (e.g., basic information, examination data, imaging data). For example, the electronic medical record can include a three-dimensional model of multiple organs and / or tissues of the patient 261.

[0095] In some embodiments, the storage device 230 can include a mass storage device, a removable storage device, a volatile read / write memory, a read-only memory (ROM), etc., or any combination thereof. In some embodiments, the storage device 230 can include a data lake and a data warehouse, which will be described in connection with Figure 3 are described in detail.

[0096] The medical service device 240 can be used to provide or assist medical services. As shown, the medical service device 240 includes a clinic terminal 240-1, a hospital bed 240-2, a smart surgery terminal 240-3, a smart nursing cart 240-4, a smart wheelchair 240-5, etc., or any combination thereof. Figure 2

[0097] The clinic terminal 240-1 refers to a terminal device configured in a clinic for use by a doctor and a patient in a medical outpatient process. For example, the clinic terminal 240-1 can include one or more of a screen, a sound output component, an image sensor, or a sound sensor. The screen of the clinic terminal 240-1 can display a consultation interface on which data can be displayed to facilitate communication between the doctor and the patient. Exemplary data can include an electronic medical record (or a portion thereof), a pre-consultation record, a medical image, a 3D organ model, a test result, a consultation suggestion, etc.

[0098] The hospital bed 240-2 refers to a bed in a hospital ward that can support an inpatient and provide user services to the patient. The hospital bed 240-2 can include a bed, a bedside terminal device, a bedside examination device, a sensor, etc., or any combination thereof. The bedside terminal device can include an XR device, a display device, a mobile device, etc., or any combination thereof. In some embodiments, the hospital bed 240-2 can be configured with an agent corresponding to inpatient services, where the hospital bed can also be referred to as a smart hospital bed or a meta-hospital bed.

[0099] The smart surgery terminal 240-3 refers to a device configured with an agent for assisting surgery. The smart surgery terminal 240-3 can perceive interactions (e.g., conversations, behaviors, etc.) between a medical service provider, a patient, and an agent, and obtain data captured by the perception device 250, thereby providing surgery assistance. In some embodiments, the smart surgery terminal 240-3 can be configured to perform risk warnings for surgical procedures, generate surgical records for surgery processes, etc., based on the agent configured therein.

[0100] ​The intelligent nursing cart 240-4 refers to a nursing cart with automatic driving function, which can assist in patient treatment and nursing. For example, the intelligent nursing cart 240-4 can be configured to guide a nurse to a hospital ward to conduct an admission examination on a patient. In some embodiments, the intelligent nursing cart can be controlled by an intelligent agent (e.g., an intelligent agent corresponding to an inpatient service, a nursing intelligent agent). In some embodiments, the intelligent nursing cart 240-4 can include a trolley, a presentation device, one or more examination devices and / or nursing tools, a perception device (such as an image sensor, a GPS sensor, a sound sensor, etc.), and the like. In some embodiments, the intelligent nursing cart 240-4 can be configured to obtain relevant treatment and nursing information of a patient, and generate examination data, nursing data, and the like. The examination data can include vital sign data of the patient. The nursing data can include a detailed record of nursing operations, such as nursing time, nursing operator, nursing measures, patient reaction, and the like.

[0101] The intelligent wheelchair 240-5 refers to a transportation device for intelligently picking up and dropping off a patient. In some embodiments, the intelligent wheelchair 240-5 can be configured to perform autonomous navigation by integrating sensors and a map (e.g., using a simultaneous localization and mapping (SLAM) technique or a pre-constructed environment model), locate a patient’s position using radio frequency identification devices (RFID), Bluetooth, or Wi-Fi signals, and identify the patient through biometric identification technology. In some embodiments, the intelligent wheelchair 240-5 can be controlled by an intelligent agent (e.g., an intelligent agent corresponding to an inpatient service, an intelligent agent corresponding to a surgical service). In some embodiments, the intelligent wheelchair 240-5 can be configured to generate data (e.g., a record of interaction content between an intelligent agent and a patient) by perceiving interaction data through a built-in camera / sensor.

[0102] The perception device 250 can be configured to collect perception information related to the environment in which it is located. In some embodiments, the perception device 250 can include perception devices in the physical hospital 110. For example, the perception device 250 can include an image sensor 250-1, a sound sensor 250-2, a temperature sensor, a humidity sensor, and the like.

[0103] The patient terminal 260 can be a terminal device that interacts with the patient 261. In some embodiments, the patient terminal 260 can include a mobile terminal 260-1, an XR device 260-2, a smart wearable device 260-3, and the like. The doctor terminal 270 can be a terminal device that interacts with the doctor 271. In some embodiments, the doctor terminal 270 can include a mobile terminal 270-1, an XR device 270-2, and the like. In some embodiments, the patient 261 can access a user space application (e.g., a patient space application) through the patient terminal 260, and the doctor 271 can access a user space application (e.g., a doctor space application) through the doctor terminal 270. In some embodiments, the patient 261 and the doctor 271 can remotely communicate with each other through the patient terminal 260 and the doctor terminal 270, thereby providing remote medical services, such as remote outpatient services, remote ward round services, remote follow-up services, and the like.

[0104] The perception device 250, the patient terminal 260, and the doctor terminal 270 can be configured as data sources to provide information to the medical service system 200. For example, these devices can transmit the collected data to the processing device 210, and the processing device 210 can provide user services based on the received data.

[0105] It should be noted that the above descriptions of the medical service systems 100 and 200 are intended to be illustrative rather than restrictive to the scope of the present application. Many alternatives, modifications, and variations will be apparent to those skilled in the art. The features, structures, methods, and other characteristics of the exemplary embodiments herein can be combined in various ways to obtain additional and / or alternative exemplary embodiments. For example, the medical service system 200 can include one or more additional components, such as terminal devices of other users, public terminal devices of a hospital, and the like. As another example, two or more components of the medical service system 200 can be integrated into a single component.

[0106] Figure 3 is a schematic diagram of an exemplary hospital support platform 300 according to some embodiments of the present application.

[0107] As shown in Figure 3 , the hospital support platform 300 can include a hardware layer 310 (also referred to as a hardware module), an interface layer 320 (also referred to as an interface module), a data processing layer 330 (also referred to as a data processing module), an application development layer 340 (also referred to as an application development module), and a service layer 350 (also referred to as a service module). It should be understood that the “layers” and “modules” in the present application are only used for logical division of the components of the hospital support platform, and are not intended to be limiting.

[0108] The hardware layer 310 can be configured to provide a hardware basis for the interaction between the real world and the digital world, and can include one or more hardware devices related to hospital operations. Exemplary hardware devices can include medical service devices, sensing devices, terminal devices, and infrastructure devices. More descriptions about hardware devices can be found elsewhere in this application. For example, see Figure 1 and related descriptions.

[0109] The interface layer 320 can be connected with the hardware layer 310 and the data processing layer 330. The interface layer 320 can be configured to obtain data collected by the hardware devices of the hardware layer 310, and send the data to the data processing layer 330 for storage and / or processing. The interface layer 320 can also be configured to control at least part of the hardware devices of the hardware layer 310. In some embodiments, the interface layer 320 can include hardware interfaces and software interfaces (e.g., data interfaces, control interfaces).

[0110] The data processing layer 330 can be configured to store and / or process data. The data processing layer 330 can include a processing device on which multiple data processing units can be configured. The data processing layer 330 can be configured to obtain data from the interface layer 320 and process the data through at least one data processing unit to implement user services related to hospital operations.

[0111] The data processing units can include various preset algorithms for implementing data processing, which can take the form of software, programs, computer codes, and / or instructions implemented by various computer programming languages (e.g., Java, C / C++). In some embodiments, the data processing layer 330 can include a processing device (e.g., the processing device 210 in Figure 2 ). The data processing units can be configured on the processing device. In some embodiments, the data processing units can include an XR unit configured to process data using XR technology to implement XR services, an AI unit configured to process data using AI technology to implement AI services (e.g., an agent unit), a digital twin unit configured to process data using digital twin technology to implement digital twin services, a data flow unit configured to process data using data flow technology (e.g., blockchain technology, data privacy computing technology) to implement data flow services, etc.

[0112] In some embodiments, the data processing layer 330 can also include a data center configured to store data. In some embodiments, the data center can employ a lake warehouse integrated architecture, which can include a data lake and a data warehouse. The data lake can be used to persist large amounts of data in a tamper-proof manner. The data warehouse can be used to store index data corresponding to the data in the data lake. The data stored in the data lake can include native (or raw) data collected by hardware devices, derived data generated based on the native data, and the like. In some embodiments, the data in the data lake can be processed by processing devices (e.g., processing devices 210).

[0113] The application development layer 340 can be configured to support application development, publishing, subscription, and the like. The application development layer 340 is also referred to as an eco-suite layer. In some embodiments, the application development layer 340 can be configured to provide an open interface for application developers to access or invoke at least a portion of the data processing unit and develop applications with at least a portion of the data processing unit. In some embodiments, as shown, the application development layer 340 can provide a development kit, an application marketplace, a multi-tenant operation platform, a cloud official website, a workspace, and other support kits to assist developers in their work. Figure 3

[0114] The service layer 350 can be configured to enable relevant users of the hospital business to access user services related to the hospital business through user space applications. More description about the user services and the user space applications can be found elsewhere in this application. For example, see Figure 1 and related description.

[0115] The present application provides a hospital support platform designed for the integrated management of various resources within a hospital, including hardware resources, software resources, and data resources. In certain embodiments, the platform further integrates data processing units capable of supporting advanced technologies, such as artificial intelligence, XR, digital twin, and blockchain. These advanced technologies are used to improve the efficiency and quality of services in the healthcare industry. For example, artificial intelligence technology enables autonomous evolution and continuous optimization of hospital operations, while XR and digital twin technology facilitates the creation and maintenance of a virtual hospital. This virtual hospital can interact with users to provide immersive and novel service experiences. In addition, the platform also includes an application development layer to grant third-party developers in the healthcare industry access to these advanced technologies. This access promotes an open ecosystem that facilitates the development and innovation of applications, thereby driving progress in medical services.

[0116] Figure 4 is a block diagram of an exemplary processing device 210 according to some embodiments of the present application. In some embodiments, the processing device 210 can be associated with a computer-readable storage medium (e.g.,​Figure 2 The processing device 210 can be configured to communicate with the storage device 230 and execute instructions stored in the computer-readable storage medium. The processing device 210 can include an obtaining module 410, a mapping module 420, and a service module 430.

[0117] The obtaining module 410 can be configured to obtain data related to a physical hospital. More description about obtaining data related to a physical hospital can be found elsewhere in this application. For example, see Figure 1 , 5 and related descriptions thereof.

[0118] The mapping module 420 can be configured to map data related to a physical hospital to a virtual hospital corresponding to the physical hospital. More description about mapping data related to a physical hospital can be found elsewhere in this application. For example, see Figure 1 , 2 and related descriptions thereof.

[0119] The service module 430 can be configured to provide user services to relevant users of a physical hospital through a user space application. More description about user services can be found elsewhere in this application. For example, see Figure 1 and 5 -7 and related descriptions thereof.

[0120] In some embodiments, the processing device 210 can include one or more other modules. For example, the processing device 210 can include a storage module for storing data generated by modules in the processing device 210. In some embodiments, any two modules can be combined as a single module, and any one module can be divided into two or more units. For example, the service module 430 can include multiple units configured to support and / or implement different user services.

[0121] Figure 5 is a schematic diagram of an exemplary flow 500 of providing user services according to some embodiments of the present application.

[0122] As shown in Figure 5 , data 530 related to a medical service flow can be collected according to a preset data collection protocol 520 corresponding to the medical service flow, and user services 540 can be provided to relevant users of the medical service flow by processing the data 530.

[0123] A medical service process refers to a process of providing a corresponding medical service to a patient. An example medical service process can include an outpatient process, an inpatient process, a surgery process, or the like, or any combination thereof. In some embodiments, a medical service process can include several standard links that a user needs to go through to obtain a corresponding medical service. For example, an outpatient process can include the following standard links, such as a registration link, a waiting link, an interrogation link, and a post-diagnosis link.

[0124] In some embodiments, medical service processes corresponding to different hospital departments of a hospital (e.g., a physical hospital 110) can be different. For example, an outpatient process corresponding to an internal department can be different from an outpatient process corresponding to a surgical department. In some embodiments, medical service processes corresponding to different diseases can be different. For example, a medical service process corresponding to a cold can be different from a medical service process corresponding to cancer.

[0125] A preset data collection protocol 520 is used to guide how to collect data 530 related to a medical service process. In some embodiments, the preset data collection protocol 520 can include data collection protocols corresponding to standard links in a medical service process. For example, a preset data collection protocol corresponding to an outpatient process can include data collection protocols corresponding to a registration link, a waiting link, an interrogation link, and a post-diagnosis link in the outpatient process.

[0126] In some embodiments, a data collection protocol corresponding to a standard link of a medical service process can at least specify one or more hardware devices for collecting data related to the standard link. The one or more hardware devices can include an Internet of Things device, a sensing device, a user terminal of a user, a public terminal of a hospital, a medical service device, and the like, or any combination thereof.

[0127] In some embodiments, a data collection protocol corresponding to a standard link can further specify data interface standards and / or data quality standards of the one or more hardware devices. A data interface standard refers to a set of guidelines and specifications that define how to exchange data between different systems, devices, or software applications. Data interface standards can include protocols, data formats, communication methods, encoding and decoding rules, security standards, and the like, or any combination thereof. By way of example only, data interface standards can include Health Level 7 (HL7), Digital Imaging and Communications in Medicine (DICOM), and the like, or any combination thereof. A data quality standard refers to a set of standards and guidelines aimed at ensuring that data is accurate, consistent, reliable, and usable for its intended purpose. Example data quality standards can include data quality model ISO / IEC 25012, data quality model ISO 8000, data standard HL7, Basel Committee on Banking Supervision (BCBS 239), and the like, or any combination thereof.

[0128] By way of example only, Figure 5 As shown, the processing device 210 can determine a preset data collection protocol 520 corresponding to the medical service process based on at least one of a hospital department 502, a disease 504, a standard procedure 506, a hardware device 508, a data interface standard 510, or a data quality standard 512 corresponding to the medical service process. In some embodiments, the preset data collection protocol 520 can be manually set by a user. In some embodiments, the agent can learn the preset data collection protocol 520 from historical data and knowledge database.

[0129] In some embodiments, the processing device 210 can update the preset data collection protocol 520 corresponding to the medical service process based on a healthcare policy 514 related to the medical service process. For example, if the healthcare policy 514 is updated to include a new standard procedure or delete some standard procedures, the processing device 210 can update the standard procedures in the preset data collection protocol 520 corresponding to the medical service process.

[0130] The data 530 related to the medical service process can include various types of data related to the medical service process. For example, the data 530 related to the medical service process can include patient location information, patient interaction information related to interactions between the patient and the patient terminal, examination data of the patient (e.g., examination report, medical image, vital sign), doctor's order, perception information related to the environment where the patient is located, patient clinical pathway, patient data input by a medical service provider (e.g., doctor, nurse) or similar personnel or any combination thereof.

[0131] In some embodiments, the processing device 210 can obtain the data 530 from the hardware device. For example, the hardware device can collect the data 530 according to the preset data collection protocol 520, and the processing device 210 can obtain the data 530 from the hardware device. In some embodiments, the processing device 210 can obtain the data 530 from a storage device (e.g., the storage device 230) storing the data 530. For example, the data 530 collected by the hardware device can be stored in the storage device, and the processing device 210 can obtain the data 530 from the storage device.

[0132] In addition, the processing device 210 can provide user services 540 to users related to the medical service process by processing the data 530. For example, at least a part of the data 530 can be processed by an agent corresponding to the medical service process. More description about the agent can be found elsewhere in this application. For example, see Figure 1 and related description. More description about processing data using an agent can be found elsewhere in this application. For example, see Figure 6 and related description.

[0133] In some embodiments, the related users can include first type users and second type users, and the user space applications can include first user space applications corresponding to the first type users and second user space applications corresponding to the second type users. For example, the first type users can be patients, and the second type users can be doctors or administrators. Accordingly, the first user space applications can be patient space applications, and the second user space applications can be doctor space applications or administrator space applications.

[0134] In some embodiments, after obtaining the interaction information between the first type user and the first user space application, the processing device 210 can provide the first user with a first user service through the first user space application based on the interaction information, and provide the second user with a second user service through the second user space application based on the interaction information. For example, when a patient is admitted to a hospital ward, the processing device 210 can provide the patient with an inpatient admission service through the patient space application, and remind a nurse of the hospital ward to pay attention to the patient through the administrator space application.

[0135] According to some embodiments of the present application, data of different patients participating in the same type of medical service process is collected through the same preset data collection protocol corresponding to the medical service process. The preset data collection protocol specifies the hardware devices for collecting data related to each standard link, data interface standard, data quality standard, etc. Compared with the traditional hospital system which only collects data in specific scenarios without considering the entire medical service process, the method disclosed herein can collect more comprehensive, more fine-grained, and more real-time medical service process data. The collected data can then be used to evaluate various aspects of the medical service process, thereby facilitating performance optimization and improving the overall efficiency of the medical service system.

[0136] Figure 6 is a schematic diagram of an exemplary process 600 of providing user services according to some embodiments of the present application.

[0137] As Figure 6 shown, after collecting data 610 related to a physical hospital, an agent 620 can process the data 610 to provide user services 640. As above, the agent 620 includes different types. According to the type and content of the data 610, the data 610 can be processed by the target type of agent 620. For example, if the data 610 is related to one link in the medical service process of a patient, the data 610 is processed by an agent corresponding to the medical service process or the link of the medical service process. For another example, the data 610 involves a specific user business, and the data 610 is processed by an agent corresponding to the user business.

[0138] In some embodiments, the processing device 210 can present a virtual person 634 corresponding to the intelligent agent 620 through a user space application 630 (e.g., a user space application), and can provide a user service 640 based on interactions between a relevant user (e.g., a patient, a hospital staff (e.g., a doctor, a nurse, a hospital administrator)) of a medical service process and the virtual person 634. In some embodiments, the relevant user can interact with the virtual person 634 through a user terminal (e.g., an XR device 632 as shown in FIG. 6B) of the user space application. For example, the user space application on the user terminal can present the virtual person to the relevant user through XR technology, and the relevant user can interact with the virtual person through the XR technology. Figure 6 The virtual person refers to a computer-generated person or entity designed to interact with a relevant user in a digital environment. The virtual person 634 can be configured to interact with the relevant user to provide the user service 640. For example, the virtual person 634 can communicate with the user by simulating human speech expressions, gestures, etc., to provide the user with a realistic communication experience. During the communication process, the user can express his / her needs or requests using natural language through voice, text, gestures, etc., and the intelligent agent 620 can be constructed based on a natural language processing algorithm to understand and analyze the user’s input, determine feedback information, and convey the feedback information to the user through the virtual person 422.

[0139] The virtual person refers to a computer-generated person or entity designed to interact with a relevant user in a digital environment. The virtual person 634 can be configured to interact with the relevant user to provide the user service 640. For example, the virtual person 634 can communicate with the user by simulating human speech expressions, gestures, etc., to provide the user with a realistic communication experience. During the communication process, the user can express his / her needs or requests using natural language through voice, text, gestures, etc., and the intelligent agent 620 can be constructed based on a natural language processing algorithm to understand and analyze the user’s input, determine feedback information, and convey the feedback information to the user through the virtual person 422.

[0140] In some embodiments, the virtual person 634 can be a digital person with certain appearance characteristics, voice characteristics, etc. For example, the processing device 210 can determine the appearance characteristics and / or voice characteristics of the virtual person 634 according to the patient data of the patient. As another example, the processing device 210 can determine the appearance characteristics and / or voice characteristics of the virtual person 634 according to the data of the doctor or nurse of the patient, so that the virtual person 634 can simulate the doctor or nurse.

[0141] Exemplary user services 640 provided to the patient based on interactions with the virtual person 634 include registration services, pre-consultation services, admission inquiry services, path guidance services, discharge services, follow-up services, etc., or any combination thereof. Some of these services can be provided by different intelligent agents. For example, the registration service can be provided by a registration intelligent agent based on interactions between the patient and a first virtual person (e.g., a nurse virtual person) representing the registration intelligent agent. As another example, the pre-consultation service can be provided by a pre-consultation intelligent agent based on interactions between the patient and a second virtual person (e.g., a nurse virtual person) representing the pre-consultation intelligent agent. As another example, the admission inquiry service can be provided by an admission intelligent agent based on interactions between the patient and a third virtual person (e.g., a doctor virtual person) representing the admission intelligent agent.

[0142] In some embodiments, the agent 620 can generate processing results 650 based on the data 610, and can provide user services 640 to relevant users based on the processing results 650. For example, the processing results 650 can include speech recognition results, image recognition results, anomaly detection results, etc. The agent 620 can utilize various AI technologies (e.g., machine learning models) to generate the processing results 650. For example, the relevant users can include hospital staff (e.g., doctors, nurses, hospital administrators), and the user services 640 provided to the hospital staff can include providing records, notifications, suggestions, etc. based on the processing results 650.

[0143] According to some embodiments of the present application, by integrating the agent, the user services can be automatically provided, so that the demand for a large amount of manpower can be reduced, the operating cost can be reduced, and the efficiency of providing the user services can be improved. In addition, the agent can continuously optimize the learned rules and mechanisms to improve the accuracy, efficiency and service quality of the user services. In addition, the present application provides different types of agents that can handle tasks of different medical service providers, different departments, different hospital positions, etc., and each agent can realize self-optimization and evolution from specific data to better handle the corresponding tasks.

[0144] Figure 7 is a flowchart of an exemplary process 700 of providing user services according to some embodiments of the present application.

[0145] At step 702, the processing device 210 can monitor updates of patient data.

[0146] The patient data can include data related to a patient participating in a medical service process. The medical service process refers to a process of providing corresponding medical services for the patient. More description about the medical service process can be found elsewhere in the present application. For example, Figure 1 , 5 and related descriptions thereof. In some embodiments, the patient data can be considered as multi-modal data, which includes multiple types of data, multi-dimensional data, etc.

[0147] In some embodiments, the data related to the patient can be collected according to a preset data collection protocol corresponding to the medical service process. For example, at least one hardware device can collect data related to the patient according to the preset data collection protocol, and the processing device 210 can obtain the data from the at least one hardware device. More description about data collection can be found elsewhere in the present application. For example, see Figure 5 and related descriptions thereof.

[0148] In some embodiments, the processing device 210 can monitor the updates of the patient-related data by monitoring the at least one hardware device. An update can be detected when the hardware device collects updated data 704 that has not been processed by the processing device 210. For example, assume that the vital sign monitor collects the patient’s heart rate every hour, the processing device 210 can detect an update when the vital sign monitor collects a new heart rate of the patient (i.e., updated data).

[0149] In some embodiments, the processing device 210 can have a direct communication connection with the at least one hardware device and directly monitor the at least one hardware device. Alternatively, the processing device 210 can have a communication connection with a storage device (e.g., the storage device 230) that stores the patient-related data collected by the at least one hardware device and monitor the at least one hardware device by monitoring the storage device.

[0150] The updated data 704 refers to the data collected by the at least one hardware device that has not been processed by the processing device 210. For example, if the at least one hardware device includes an Internet of Things device, the updated data 704 can include Internet of Things data. The updated data 704 can be obtained directly from the at least one hardware device or from a storage device.

[0151] If the patient-related data includes the updated data 704 obtained from the at least one hardware device, the processing device 210 can perform an event of interest (EOI) detection 706 on the updated data. Meanwhile, the processing device 210 can continue to perform the step 702 to continuously monitor the updates in the patient data.

[0152] In some embodiments, the processing device 210 can perform the EOI detection 706 on the updated data 704.

[0153] The EOI refers to a specific event or action that needs attention. The EOI detection 706 refers to processing the updated data 704 collected by the at least one hardware device to detect whether one or more EOIs occur. For example, for an inpatient procedure, example EOIs can include the patient checking into a hospital room, conducting an admission check on the patient, at least one doctor making a round in the hospital room, performing a care operation or a medical check on the patient, the patient initiating a service request, obtaining or updating a doctor’s order for the patient, an abnormal physiological state of the patient, the doctor issuing an instruction to the patient. As another example, for a surgery procedure, example EOIs can include a surgery participant issuing an instruction (e.g., a lead surgeon issuing an instruction for the patient to receive a blood transfusion), detecting a surgery risk (e.g., the patient’s heart stopping or severe bleeding), an abnormal physiological state of the patient (e.g., the patient’s blood pressure being too low), a count of a surgery tool being less than a threshold (e.g., a count of a scalpel being less than a scalpel count threshold), completion of the surgery (i.e., an end of the surgery procedure), etc., or any combination thereof.

[0154] In some embodiments, the processing device 210 can perform the EOI detection 706 based on EOI detection rules 712. The EOI detection rules 712 refer to rules that need to be followed when performing the EOI detection 706. For example, the EOI detection rules 712 can specify the EOI that need to be detected, the algorithm or technique used to detect a specific EOI, the algorithm or technique used to analyze the data collected from a specific data source, the EOI corresponding to different stages of a hospitalization procedure, the EOI corresponding to different types of patients, etc., or any combination thereof.

[0155] In some embodiments, the EOI detection rules 712 can be determined based on the history of EOI detection, or manually set by a user (e.g., a doctor, a nurse, a technician, etc.). For example, the processing device 210 can perform the EOI detection 706 on the update data 704 based on the type of the update data 704.

[0156] In some embodiments, each stage of a medical service procedure can correspond to one or more EOIs, and different stages of a medical service procedure can correspond to different types of EOIs. Therefore, the processing device 210 can perform the EOI detection 706 on the update data 704 based on the current stage of the patient in the medical service procedure. For example, the processing device 210 can determine the current stage of the patient in the medical service procedure, and determine the type data of the EOI that need to be detected according to the current stage of the patient. In addition, the processing device 210 can perform the EOI detection 706 on the update data 704 based on the type data.

[0157] By performing the EOI detection based on the current stage of the patient in the medical service procedure, only specific types of EOIs need to be detected or tracked, which can reduce the amount of data to be processed, thereby reducing the processing requirement of the processing device 210 and improving the efficiency of the EOI detection.

[0158] If there is an EOI occurrence 708, the processing device 210 can perform one or more preset operations 710 corresponding to the EOI to provide user services to the relevant user. Meanwhile, the processing device 210 can continue to perform the step 702 and the EOI detection 706 to continuously monitor the data update and detect the EOI.

[0159] In some embodiments, the processing device 210 can perform one or more preset operations 710 corresponding to the EOI to provide user services to the relevant user.

[0160] The relevant users can include any user related to the medical service process. For example, the users can include a patient (or a part of the patient (e.g., an organ)), an accompanying person, a visitor of the patient, a hospital staff of the medical service process (e.g., a medical service provider (e.g., a doctor, a nurse, a technician, etc.) of the medical service process, a hospital administrator, a support staff, etc.), a supplier of the medical service process, an application developer of the medical service process, etc., or any combination thereof.

[0161] The user services can include any service provided in the medical service process. For example, the user services include medical services corresponding to the medical service process provided to the patient and / or the accompanying person, support services corresponding to the medical service process provided to the hospital staff and / or the supplier, etc.

[0162] The preset operations corresponding to the EOI refer to operations that need to be performed when the EOI occurs. In some embodiments, the preset operations can include general operations and / or specific operations. The general operations refer to operations that need to be performed as long as the EOI occurs, regardless of the type of the EOI. For example, the general operations can include generating a record related to the EOI. The specific operations refer to operations that are performed when a specific type of EOI occurs. For example, updating the daily plan of the patient according to the updated data can be determined as a specific operation corresponding to the EOI of updating the medical order of the patient by the doctor.

[0163] In some embodiments, the processing device 210 can determine the correspondence 714 between the EOI and the preset operations, and determine one or more preset operations 710 corresponding to the detected EOI based on the correspondence 714. The correspondence 714 can indicate that one or more preset operations 710 need to be performed when a specific type of EOI occurs. For example, the correspondence 714 can take the form of a query table.

[0164] In some embodiments, the correspondence 714 can be determined in advance and stored in a storage device, and the processing device 210 can obtain the correspondence 714 from the storage device. In some embodiments, the processing device 210 can determine the correspondence 714 between the EOI and the preset operations according to historical records.

[0165] In some embodiments, one or more preset operations 710 corresponding to the EOI can be further determined according to the profile data of the patient. The profile data of the patient can include basic data, health data, historical data, registration data, etc., or any combination thereof. For example, different types of patients (e.g., suffering from different diseases, having different ages) can correspond to different predetermined operations.

[0166] In some embodiments, the preset operation corresponding to the EOI can be performed immediately after the EOI is detected. For example, the processing device 210 can cause the nurse terminal to provide an alarm when the physiological state of the patient is abnormal. In some embodiments, the preset operation corresponding to the EOI can be performed after the EOI is completed. For example, the record related to the EOI can be generated in response to detecting the completion of the EOI.

[0167] According to some embodiments of the present application, at least one hardware device that captures information about a medical service process is monitored. Such monitoring helps to timely discover the occurrence of data updates and EOIs, and timely trigger the corresponding preset operations. Therefore, user services can be automatically and efficiently provided to relevant users, thereby improving service efficiency and quality. In addition, the monitored hardware device collects multi-modal data at different links of the entire medical service process, thereby realizing patient-centered healthcare and comprehensive hospital services.

[0168] In some embodiments, the processing device 210 can configure an agent corresponding to the medical service process, and the agent can perform at least part of the process 700. An agent refers to an agent that acts in an intelligent manner. For example, the agent can learn an EOI detection rule 712 from historical records, and perform EOI detection according to the EOI detection rule. The EOI detection rule refers to a rule for performing EOI detection on updated data. For example, the agent can learn from historical records which types of EOIs need to be monitored, how to effectively detect EOIs, which data needs to be analyzed to detect EOIs, etc. For another example, the agent can learn from historical records the correspondence between EOIs and preset operations, and determine one or more preset operations corresponding to the EOI according to the correspondence. For example, the agent can learn from historical records which operations need to be performed when the EOI occurs.

[0169] In some embodiments, the agent can further learn the EOI detection rule and / or the correspondence between the EOI and the preset operation based on profile data of different patients. For example, the agent can determine different EOI detection rules and different preset operations for different types of patients (e.g., different diseases, different ages).

[0170] By integrating the agent, the system can continuously learn the EOI detection rule and / or the corresponding relationship by using big data technology, machine learning technology, and other advanced methods. Such continuous optimization of the EOI detection rule and / or the corresponding relationship improves the accuracy, efficiency, and service quality of inpatient services.

[0171] Figure 8 is a schematic diagram of an example medical outpatient process 800 according to some embodiments of the present application. As shown in FIG. 8, the medical outpatient process 800 includes the following steps: Figure 8As shown, the medical outpatient process 800 can include a registration link 810, a waiting link 820, an inquiry link 830, and a post-diagnosis link 840, and the like. In some embodiments, the processing device 210 (e.g., the service module 430, the agent corresponding to the outpatient service / process configured on the processing device 210) can perform operations involved in multiple links of the medical outpatient process 800.

[0172] The patient can make an appointment at the registration link 810. As shown, the patient can make an appointment through a patient terminal or a registration terminal on site. Figure 8 As shown, the operations related to the registration link 810 can include obtaining / establishing an electronic medical record, providing intelligent registration services, providing path planning services, providing path guidance services, and the like. In some embodiments, the patient can initiate a registration request through a patient terminal or a registration terminal on site, or initiate a remote registration request through a patient terminal at other places outside the hospital.

[0173] In some embodiments, after receiving the registration request from the patient, the processing device 210 can obtain or establish the patient's electronic medical record.

[0174] The electronic medical record refers to an electronic file that records various types of patient data. In some embodiments, the electronic medical record can be updated as the patient's outpatient process progresses. For example, when the patient registers, the electronic medical record can include the patient's basic information; after the patient completes the registration, the electronic medical record can be updated to further include the patient's chief complaint and registration record.

[0175] In some embodiments, when receiving the registration request from the patient, the processing device 210 can provide intelligent registration services for the patient. The intelligent registration service can be used to match the registration department, the registration doctor, and the registration time for the patient. In some embodiments, the processing device 210 can provide services related to the registration link through the registration terminal of the hospital or the patient terminal of the patient.

[0176] If the patient does not have a specific department for registration, the processing device 210 can provide intelligent registration services for the patient based on the patient's chief complaint, the patient's uploaded examination report, and / or the historical patient information stored by the storage device, to determine the recommended department matching the patient's visit needs. If the patient does not have a specific doctor for registration, the processing device 210 can recommend a registration doctor for the patient based on the patient's chief complaint, and / or show the doctors of the registration department to the patient. The processing device 210 registration module 410 can recommend a registration time period with fewer appointments for the patient based on the appointment information of the registration doctor.

[0177] In some embodiments, the processing device 210 can determine the registration doctor of the patient by using the patient terminal or the registration terminal to conduct a first inquiry on the patient. For example, the processing device 210 can obtain the patient's chief complaint through the patient terminal or the registration terminal, and determine at least one candidate department according to the patient's chief complaint. Then, the processing device 210 can cause the patient terminal or the registration terminal to conduct a first inquiry on the patient based on the at least one candidate department. For example, the processing device 210 can determine the first inquiry content of the first inquiry according to the patient's chief complaint and the at least one candidate department, and cause the patient terminal or the registration terminal (e.g., an XR device) to present a first virtual character that conducts the first inquiry on the patient based on the first inquiry content. Finally, the processing device 210 can determine the doctor according to the first data collected by the patient terminal or the registration terminal in the first inquiry.

[0178] The path planning service can be used to provide the patient with a planned path to the registration doctor's clinic. Specifically, the processing device 210 can generate a planned path to the clinic according to the current location of the patient and the location information of the registration doctor's clinic. In some embodiments, the processing device 210 can generate the planned path after the patient registers on site at the hospital. In some embodiments, the processing device 210 can generate the planned path when the patient registers remotely and arrives at the hospital.

[0179] The path guidance service can be used to guide the patient to the registration doctor's clinic according to the planned path. In some embodiments, the processing device 210 can display guidance information to the patient through the patient's patient terminal (e.g., an XR device). For example, through AR technology, the XR device can overlay guidance information related to the planned path on the patient's real view.

[0180] The patient can wait and prepare for the visit in the waiting stage. In the waiting stage 820, the patient can be on site at the hospital (e.g., in the waiting area in front of the registration doctor's clinic), or at another location outside the hospital (e.g., at the patient's home). As shown, the operations related to the waiting stage 820 can include providing pre-visit services, providing force feedback / temperature feedback, etc. Figure 8 The pre-visit service can be used to collect information about the patient by conducting a preliminary inquiry on the patient before the patient enters the clinic for formal visit. Specifically, when the patient is waiting for the visit, the processing device 210 can conduct a pre-visit on the patient through the patient's patient terminal or a waiting terminal configured in the waiting area, to alleviate the anxiety of the patient when waiting, generate a pre-visit record, and provide the pre-visit record for the doctor's reference, thereby improving the doctor's visit efficiency.

[0181] The pre-visit service can be used to collect information about the patient by conducting a preliminary inquiry on the patient before the patient enters the clinic for formal visit. Specifically, when the patient is waiting for the visit, the processing device 210 can conduct a pre-visit on the patient through the patient's patient terminal or a waiting terminal configured in the waiting area, to alleviate the anxiety of the patient when waiting, generate a pre-visit record, and provide the pre-visit record for the doctor's reference, thereby improving the doctor's visit efficiency.

[0182] The force feedback / temperature feedback can be used to provide soothing feedback to the patient. Specifically, when the patient is waiting, the processing device 210 can apply force feedback and / or temperature feedback to the patient through the wearable device of the patient after detecting the emotion (e.g., nervousness, fear, anxiety, etc.) of the patient, so that the patient feels the action of handshaking, hugging, etc., thereby soothing the bad mood of the patient.

[0183] The patient can communicate with the registration doctor in the inquiry session 830 to receive a medical outpatient service (e.g., an on-site outpatient service in a clinic room, a remote outpatient service). As shown, the operations related to the inquiry session 830 can include a daily outpatient preview, a medical data display, a provision of a consultation suggestion, a diagnosis record generation, a remote accompanying service, etc. Figure 8

[0184] In the inquiry session 830, the daily outpatient preview is used to show the doctor the relevant information of the patients to be treated on the day. For example, the processing device 210 can show the doctor the categories of the patients registered or waiting on the day before the doctor starts the consultation. The categories of the patients can include new patients, return patients, etc. In some embodiments, the doctor can browse the daily outpatient preview through a public terminal device in the clinic room or the doctor terminal (e.g., an XR device) of the doctor.

[0185] In the inquiry session 830, the medical data display is used to display the medical data (e.g., an electronic medical record) of the patient to the doctor, the patient, and / or the remote accompanying person. Specifically, the processing device 210 can synchronously display the medical data of the patient to the doctor, the patient, and / or the remote accompanying person through at least one terminal device. In addition, the display manner and / or the display content of the electronic medical record are updated according to the interactive operation of the doctor, the patient, and / or the remote accompanying person on the medical data, so as to facilitate the communication among the doctor, the patient, and / or the remote accompanying person.

[0186] In the inquiry session 830, the provision of the consultation suggestion refers to providing the consultation suggestion that can be referred to by the doctor during the consultation. Specifically, the processing device 210 can generate the consultation suggestion provided to the doctor based on the perception information collected by the perception device during the consultation, so as to facilitate the doctor to adjust the diagnosis manner, the diagnosis result, and the prescription, thereby improving the efficiency and accuracy of the consultation.

[0187] The diagnosis record generation is used to assist the doctor to generate the diagnosis record of the patient. Specifically, the processing device 210 can generate the diagnosis record to record the information about the medical condition of the patient diagnosed by the doctor, the medical order of the doctor, etc.

[0188] ​The remote accompanying service can be used to provide immersive accompanying service for the patient and the remote accompanying person. For example, the processing device 210 can show the virtual clinic room simulating the real clinic room, the real-time picture of the doctor in the clinic room, and the real-time picture of the patient to the remote accompanying person. As another example, the processing device 210 can present the real-time image of the remote accompanying person through the public terminal device in the clinic room. As another example, the processing device 210 can present the real-time image of the remote accompanying person to the doctor and the patient through the doctor terminal and the patient terminal of the doctor and the patient, respectively.

[0189] More descriptions about the medical data display, the provision of the consultation suggestion, the generation of the diagnosis record, and the remote accompanying service can be found in the related descriptions of Figure 11 , which are not repeated here.

[0190] In some embodiments, the processing device 210 can provide services related to the consultation session to the relevant users (e.g., the doctor, the patient, the remote accompanying person) through at least one terminal device. The at least one terminal device includes the clinic room public terminal device, the patient terminal, the doctor terminal, the remote accompanying person terminal device, and the like.

[0191] The public terminal device in the clinic room refers to the terminal device installed on the clinic site, which can include a display screen, a sound output device, a sound sensor, an XR device, a wearable device, and the like, or any combination thereof. For example, after the patient enters the clinic room, the processing device 210 can present the electronic medical record, the decision suggestion, the pre-consultation record, the diagnosis record, the real-time picture of the remote accompanying person, and the like to the doctor and / or the patient through the display screen of the public terminal device. As another example, the patient can wear the wearable device of the public terminal device, and receive the force feedback of the remote accompanying person through the wearable device. As another example, the doctor and the patient can wear the XR device of the public terminal device, and the processing device 210 can present various types of information to the doctor and / or the patient through the XR device of the public terminal device. In some embodiments, the XR device worn by the patient can also be the XR device of the registration terminal.

[0192] Reference is made to Figure 12 , Figure 12 is a schematic diagram of an example consultation interface 1200 according to some embodiments of the present application. As shown in Figure 12 , the consultation interface 1200 can include a first interface element 1210 related to the electronic medical record of the patient, a second interface element 1220 related to the remote accompanying service, a third interface element 1230 related to the medical document, and a fourth interface element 1240 related to the consultation suggestion.

[0193] The presentation content and / or presentation form of the consultation interface 1200 can change as the consultation process proceeds. For example, the doctor and / or the patient can issue control instructions to adjust the content displayed in the consultation interface 1200. The control instructions can include various forms of instructions, such as voice control instructions, gesture control instructions, touch control instructions, and control instructions input through an input device (e.g., a mouse, a keyboard, etc.).

[0194] The first interface element 1210 can be an icon corresponding to the electronic medical record, or can be used to display the content of the electronic medical record. During the consultation process, the doctor and / or the patient can call up at least a portion of the electronic medical record by issuing control instructions to present it through the first interface element 1210.

[0195] The second interface element 1220 can be an icon corresponding to the remote accompanying service, or can be used to display the picture of the remote accompanying service. When the patient's remote accompanying request is approved by the doctor, the remote accompanying person can be communicated with during the consultation process, and the picture of the remote accompanying person can be viewed through the second interface element 1220.

[0196] The third interface element 1230 can be an icon corresponding to the medical document, or can be used to display the medical document, such as the pre-consultation record, the diagnosis record (the initial diagnosis record or the target diagnosis record), etc. When it is detected that the patient starts the consultation, the pre-consultation record can be presented through the third interface element 1230; when the doctor and the patient complete the communication, the third interface element 1230 can be updated to present the initial diagnosis record; when the doctor confirms the modification of the initial diagnosis record, the third interface element 1230 can be updated to present the target diagnosis record.

[0197] In some embodiments, the processing device 210 can determine that the patient starts the consultation based on a call instruction. The call instruction is an instruction that the doctor indicates to start the next patient's consultation. The doctor can actively initiate the call instruction through the public terminal or the doctor terminal. Alternatively, the processing device 210 can automatically generate the call instruction after the doctor submits the target diagnosis record of the previous patient. In some embodiments, when the patient receives the on-site medical outpatient service, the processing device 210 can determine that the patient starts the consultation based on the positioning information of the patient terminal. Alternatively, the processing device 210 can determine that the patient starts the consultation based on the image of the patient entering the consultation room obtained by the sensing device (e.g., an image sensor). When the patient receives the remote medical outpatient service, the processing device 210 can obtain the remote consultation confirmation messages sent by the doctor and the patient through the doctor terminal and the patient terminal respectively to determine that the patient starts the consultation.

[0198] The fourth interface element 1240 can be an icon corresponding to the consultation suggestion, or be used to present the content of the consultation suggestion. During the medical process, after the consultation suggestion is generated based on the perception information, the consultation suggestion can be presented through the fourth interface element 1240.

[0199] In some embodiments, the processing device 210 can provide services related to the consultation session to the doctor and the patient through a doctor terminal of the doctor and a patient terminal of the patient. At least part of the content presented by the doctor terminal and the patient terminal can be synchronized.

[0200] In some embodiments, the processing device 210 can further provide services related to the consultation session to the remote accompanying doctor through a remote terminal device (for example, an XR terminal device) of the remote accompanying doctor. When the remote accompanying doctor participates in the medical process, the remote terminal device of the remote accompanying doctor can synchronously present the virtual consultation room.

[0201] In some embodiments, the processing device 210 can provide remote medical outpatient services. For example, the processing device 210 can obtain or generate a three-dimensional patient model. The 3D patient model can correspond to the patient or a part (for example, the upper body) of the patient. For example only, the processing device 210 can obtain an initial three-dimensional patient model from the electronic medical record of the patient, and can update the initial three-dimensional patient model based on real-time dynamic data and physiological data of the patient, thereby obtaining the three-dimensional patient model. In addition, the processing device 210 can present the three-dimensional patient model to the doctor, and obtain the examination instructions input by the doctor through the doctor terminal. Specifically, the processing device 210 can present the three-dimensional patient model on the field of view of the doctor through the doctor terminal. The examination instructions input by the doctor can include examination parts, examination devices, and examination operations. The doctor can input the examination instructions in various ways, such as voice, gestures, and operation input devices (for example, smart gloves, smart grips, etc.). For example, the doctor terminal can present virtual examination devices corresponding to a plurality of examination devices. The doctor can select a virtual examination device through an input device, and perform a virtual examination operation on the three-dimensional patient model using the virtual examination device. The processing device 210 can determine the examination parts, the examination devices, the examination operations, etc. according to the virtual examination operation performed by the doctor, and generate the examination instructions. In addition to the examination instructions, the doctor can also input other instructions to instruct the doctor terminal to rotate, zoom in, zoom out, etc. the three-dimensional patient model.

[0202] In some embodiments of the present application, the doctor and the patient can remotely communicate in the virtual consultation room space through the doctor terminal and the patient terminal, which can provide the doctor and the patient with an immersive medical experience without barriers. In addition, the three-dimensional patient model of the patient can also be presented to the doctor, so that the remote examination can be more accurate and convenient, thereby improving the accuracy of diagnosis.

[0203] After the patient completes the medical treatment, the post-medical treatment session can be entered. For example,Figure 8 As shown, the operations related to the post-diagnosis session 840 can include providing a medication service, providing an examination service, providing a health monitoring service, etc.

[0204] The medication service is used to assist the patient to take the medicine prescribed by the doctor. For example, the medication service is used to assist the patient to pay for the medicine, make an appointment with the pharmacy for taking the medicine, guide the patient to the pharmacy, etc. The examination service is used to assist the patient to take the examination required by the doctor. For example, the examination service is used to assist the patient to pay for the examination, make an appointment with the examination department, guide the patient to the examination department, etc.

[0205] In some embodiments, in response to detecting that the consultation process is over, the processing device 210 can determine a target service to be provided to the patient after the consultation process, and make an appointment for the patient with a target business unit providing the target service. The processing device 210 can detect whether the consultation process is over in various ways. For example, when it is detected that the doctor has submitted a target diagnosis record of the patient or the doctor has called the next patient, the processing device 210 can determine that the consultation process of the patient is over.

[0206] In some embodiments, the processing device 210 can determine the target service required by the patient based on the target treatment prescription in the target diagnosis record. For example, the target service can be a medication service, and the target business unit can be a pharmacy. When the patient requires the medication service, the processing device 210 can send the medicine prescription information of the patient to the pharmacy, and make an appointment for taking the medicine. For another example, the target service can be an examination service, and the target business unit can be an examination department. When the patient requires the examination service, the processing device 210 can send the examination prescription information of the patient to the examination department, and make an appointment for the examination. Optionally, after making the appointment with the target business unit, the processing device 210 can send the appointment information to the patient terminal. The health monitoring service can be used to continuously monitor the health condition of the patient after the medical outpatient service process. For example, the processing device 210 can generate a health monitoring plan based on the target diagnosis record, and cause one or more monitoring devices to obtain health monitoring information of the patient based on the health monitoring plan. Optionally, the processing device 210 can further update the health monitoring plan based on the health monitoring information.

[0207] In some embodiments, one or more operations in the medical outpatient service process 800 can be performed by an agent corresponding to the medical outpatient service / process. In some embodiments, different sessions of the medical outpatient service process 800 can share one agent, or different sessions can correspond to different agents.

[0208] Figure 9 is a schematic diagram of an example process 900 of providing a pre-consultation service according to some embodiments of the present application. Figure 9 As shown, the process 900 can be performed in the waiting session 820.

[0209] At step 910, the second inquiry content of the second inquiry is determined based on the department of the doctor.

[0210] The second inquiry can also be referred to as a pre-consultation inquiry, which is used to preliminarily inquire the patient before formal consultation. The second inquiry can include multiple rounds of inquiries. The second inquiry content can include inquiry content of each round of inquiry. Alternatively, the second inquiry content only includes inquiry content of the first round of inquiry.

[0211] In some embodiments, the processing device 210 can obtain a pre-consultation record template corresponding to the department of the doctor, and determine the second inquiry content according to the pre-consultation record template.

[0212] In some embodiments, the processing device 210 can obtain known information about the patient (e.g., electronic medical record, chief complaint, etc.), and determine missing information in the pre-consultation record template that has not been collected by comparing the pre-consultation record template and the known information. For example, if the known information includes the family history of the patient, the missing information does not need to include the family history. For another example, if the patient's complaint includes the patient's medical history, the missing information does not need to include the medical history. In some embodiments, the missing information can be determined based on the basic information of the patient. For example, for a male patient, the missing information does not need to include menstrual history and fertility history. In addition, the processing device 210 can determine the second inquiry content based on the missing information.

[0213] In some embodiments, the processing device 210 can use a first inquiry model to determine the second inquiry content based on the department of the doctor and the known information about the patient. The first inquiry model can include a CNN model, an RNN model, an LSTM model, a BERT model, a ChatGPT model, etc. In some embodiments, the first inquiry model can include a missing information determination model and a first inquiry content determination model. The missing information determination model can be configured to output missing information by processing the department of the doctor and the known information about the patient. The first inquiry content determination model can be configured to output the second inquiry content based on the missing information of the patient.

[0214] At step 920, the patient terminal is controlled to conduct a second inquiry on the patient based on the second inquiry content.

[0215] In some embodiments, after the patient makes an appointment with the doctor, the processing device 210 can determine an estimated waiting time for the patient to receive a medical consultation service. For example, the estimated waiting time can be a time difference between the current time and the appointment time period of the patient. For another example, the estimated waiting time can be determined based on the daily outpatient record of the doctor and the appointment record of the patient. The appointment record of the patient can include an appointment time period appointed by the patient. The daily outpatient record of the doctor is a record reflecting the daily outpatient situation of the doctor.

[0216] In some embodiments, in response to determining that the estimated waiting time is greater than the first preset time threshold, the processing device 210 can cause the patient terminal of the patient to initiate a second inquiry to the patient or display a suggestion to make the second inquiry. In this way, sufficient time can be ensured for the pre-consultation, and the doctor can be prevented from calling the patient during the pre-consultation process.

[0217] In some embodiments, in response to determining that the estimated waiting time is less than a second preset time threshold, the processing device 210 can cause the patient terminal of the patient to initiate a second inquiry to the patient or display a suggestion to make the second inquiry. The second preset time threshold can be greater than the first preset time threshold. For example, when it is detected that the time from the current time to the registration time period is less than 24 hours (i.e., the estimated waiting time is less than 24 hours), the patient terminal can display a suggestion to make the second inquiry (such as by a virtual character to the patient) to the patient, thereby timely reminding the patient to make the pre-consultation.

[0218] In some embodiments, the processing device 210 can detect that the patient initiates a pre-consultation request through the patient terminal, and then cause the patient terminal of the patient to make a second inquiry to the patient.

[0219] In some embodiments, the patient terminal can present a second virtual character to make a second inquiry based on the second inquiry content. The second virtual character refers to a digital character image with specific characteristics (e.g., specific appearance characteristics, voice characteristics, etc.), and can communicate with the patient to make a pre-consultation. Specifically, the processing device 210 can display the second virtual character through the screen of the patient terminal (e.g., an XR device), and play the second inquiry content through the sound output device of the patient terminal. At the same time, the second virtual character can simulate human speech expression, gestures, etc., to provide a realistic communication experience for the patient.

[0220] In some embodiments, the second virtual character can have preset appearance characteristics. In some embodiments, the appearance characteristics of the second virtual character can be determined based on the optical image data of the doctor registered by the patient. In some embodiments, the appearance characteristics of the second virtual character can be determined according to the basic information of the patient. In some embodiments, the processing device 210 can select a suitable virtual character as the second virtual character from a virtual character library based on the appearance characteristics of the doctor and / or the basic information of the patient.

[0221] In some embodiments, the second inquiry includes multiple rounds of inquiries, and the second inquiry content can include inquiry content of each round of inquiry in the second inquiry, and the second inquiry can be performed through Figure 10 as shown in the flow 1000.

[0222] As Figure 10As shown, for the first round of interrogation, the processing device 210 can cause the patient terminal to conduct the first round of interrogation based on the corresponding interrogation content.

[0223] For each current round of interrogation (referred to as current interrogation) other than the first round of interrogation, the processing device 210 can adjust the interrogation content of the current interrogation (referred to as current interrogation content) based on the second data collected before the current interrogation, so as to make the interrogation content more in line with the patient's condition. Specifically, the processing device 210 can determine the semantic information and emotional information of the patient's historical answer based on the second data collected before the current interrogation. The second data can be collected by detecting the sound by the sound sensor of the third terminal device. The historical answer is the answer of the patient to the interrogation of the historical round. The semantic information of the historical answer represents the content of the historical answer. The emotional information of the historical answer can indicate the emotion (e.g., calm, nervous, anxious, scared, suspicious, irritable, etc.) of the patient when providing the historical answer. The processing device 210 can determine the semantic information by performing text transcription, speech content recognition, etc. on the second data. The processing device 210 can determine the emotional information by analyzing the features such as content, tone, intonation, speed, etc. of the second data.

[0224] With reference to the foregoing Figure 10 , the processing device 210 can adjust the current interrogation content based on the semantic information and the emotional information. For example, when the emotional information of the patient is "nervous" or "scared", the processing device 210 can add soothing words in the current interrogation content. For another example, when the semantic information indicates that the patient does not answer the historical interrogation clearly, the processing device 210 can adjust the current interrogation content to repeat the historical interrogation, so as to guide the patient to answer the historical interrogation clearly. The originally determined current interrogation content can be used as the interrogation content of the next round of interrogation. In this way, the current interrogation content can be adjusted in time according to the patient's condition, so as to improve the service quality of the pre-consultation.

[0225] In some embodiments, in addition to adjusting the current interrogation content, the sound features used for interrogation can also be adjusted in real time based on the state of the patient. The sound features include speed features, tone features, intonation features, volume features, etc. As shown, the processing device 210 can determine the sound features of the current interrogation according to the semantic information and emotional information of the historical answer of the patient, and cause the patient terminal to conduct the current interrogation according to the adjusted interrogation content and the sound features of the current interrogation. In this way, the emotional changes of the patient can be better taken care of, so as to enhance the personification effect of the second virtual character and improve the quality of the pre-consultation service. Figure 10

[0226] In some embodiments, as shown Figure 10 ​As shown, the processing device 210 can further obtain the patient's physiological state information. The patient's physiological state information can reflect the patient's real-time physiological state. The physiological state information may include the patient's physiological parameter values ​​(e.g., heart rate, pulse rate, respiratory rate, etc.). The physiological state information may also include information related to the patient's posture, limb behavior, facial expressions, muscle state, etc. In some embodiments, the patient's physiological state information can be obtained using a wearable device worn by the patient.

[0227] Furthermore, the processing device 210 can adjust the current inquiry content based on semantic information, emotional information, and physiological state information. Specifically, the processing device 210 can update the patient's emotional information based on the patient's physiological state information. It is understood that a patient's inner emotions are not always fully expressed through their answers; therefore, the patient's emotional information may be updated or modified based on the patient's physiological state information. In addition, the processing device 210 can adjust the current inquiry content based on semantic information and updated emotional information.

[0228] According to some embodiments of this application, by further considering the patient's physiological state data, the accuracy of the patient's emotional information can be improved, thereby improving the accuracy of adjusting the current inquiry content and thus improving the service quality of the pre-diagnosis service.

[0229] like Figure 10 As shown, in some embodiments, the processing device 210 can determine feedback parameters based on at least a portion of semantic information, emotional information, and physiological state information, and control the wearable device to apply feedback to the patient according to the feedback parameters. Feedback may include at least one of force feedback or temperature feedback. Feedback parameters can be used to control the manner in which feedback is applied, such as the type of feedback, the body part to which feedback is applied, and the intensity of feedback. In some embodiments, the processing device 210 can determine the patient's mood and mood level based on at least a portion of semantic information, emotional information, and physiological state information, and determine feedback parameters based on the mood and mood level. This method can promptly soothe the patient's negative emotions, thereby improving the quality of pre-consultation services.

[0230] In some embodiments, the processing device 210 may terminate the second query based on a second preset condition. The second preset condition may be that the number of remaining missing information is 0. The second preset condition may be that the time difference between the patient's current time and the estimated waiting time is less than a threshold. The second preset condition may also be similar to the first preset condition.

[0231] In some embodiments, the second inquiry content determined in step 910 can only include the inquiry content of the first inquiry. The current inquiry content of each current inquiry in the first inquiry can be determined during the process of the second inquiry. For example, in the current inquiry, the processing device 210 can input the inquiry content of the historical inquiry, the historical answer of the patient, the known information of the patient, etc. into the second inquiry content determination model, and output the current inquiry content from the second inquiry content determination model.

[0232] In step 930, a pre-consultation record is generated based on the second data collected by the patient terminal in the second inquiry.

[0233] The second data can include voice data, text data and image data input by the patient terminal in the second inquiry. The pre-consultation record can be used to record the patient information collected in the second consultation (i.e. pre-consultation). Optionally, some known information of the patient can also be recorded in the pre-consultation record. In some embodiments, the pre-consultation record is generated according to a preset template. The preset template can be the template corresponding to the department of the doctor, or the template set by the doctor.

[0234] For example, when the second data includes voice signals, the processing device 210 can first transcribe the voice signals into text, and extract the second keywords from the text by a keyword extraction algorithm. In addition, the processing device 210 can convert the second keywords into medical terms. In addition, the processing device 210 can obtain a plurality of template fields in the pre-consultation record template, retrieve the content corresponding to each template field from the hospital terms and fill in the corresponding position of the pre-consultation record template. The conversion of the second keywords can be based on a term conversion model, or based on a knowledge dictionary.

[0235] In some embodiments, the second inquiry can be performed by a terminal device other than the patient terminal (e.g. a waiting terminal). In some embodiments, the flow 900 can be executed by an intelligent agent corresponding to a medical outpatient service or a medical outpatient process. For example, the pre-consultation service can be provided by a pre-consultation intelligent agent.

[0236] Figure 11 FIG. 11 is a flowchart of an example flow 1100 for providing a medical outpatient service based on perception information according to some embodiments of the present application. In some embodiments, the flow 1100 can include one or more of sub-flows 1110, 1120, 1130 and 1140.

[0237] The sub-flow 1110 can be used to provide a consultation suggestion based on the perception information. The sub-flow 1110 can be executed in the consultation step 830. As shown, the sub-flow 1110 can include steps 1112 and 1114. Figure 11

[0238] ​At step 1112, an appointment suggestion can be generated based on the perception information and the patient data. The appointment suggestion refers to a suggestion to assist the doctor to provide the medical appointment service. An example of the appointment suggestion can include a supplementary inquiry suggestion, a physical examination suggestion, a prescription suggestion, a treatment plan suggestion, and the like.

[0239] In some embodiments, the appointment suggestion can be determined based on a knowledge database corresponding to the registered department, an appointment specification, and the like. For example, the processing device 210 can determine the conversation content between the doctor and the patient based on the voice signal collected by the sound sensor, and search in the knowledge database, the appointment specification, and the like based on the conversation content and / or the patient data to determine the appointment suggestion. For example only, for example only, a search can be performed in the appointment specification based on the conversation content and / or the patient data to determine which information in the appointment specification has not been collected, and a supplementary inquiry suggestion is provided based on the information.

[0240] In some embodiments, the appointment suggestion can be generated based on a diagnosis model. Specifically, the appointment module 430 can determine a model input based on the perception data, the perception information, and the patient data, and input the model input to the diagnosis model, which can output a corresponding appointment suggestion. For example, the model input can include the patient data, the communication content determined based on the voice signal, the patient state information determined based on the image data, or any combination thereof.

[0241] In some embodiments, the appointment suggestion can be generated by an agent corresponding to the medical appointment service. The agent can learn a generation mechanism for generating the appointment suggestion from various data (e.g., historical diagnosis records, a knowledge database, and an appointment specification), and process the perception information and the patient data according to the mechanism to provide the appointment suggestion.

[0242] At step 1114, at least one part of the at least one terminal device can be controlled to present the appointment suggestion.

[0243] For example, when the patient receives the on-site medical appointment service in the clinic, the processing device 210 can control the public terminal device or the doctor terminal to present the appointment suggestion. For another example, when the patient receives the remote medical appointment service, the processing device 210 can control the doctor terminal of the doctor and the patient terminal of the patient to present the appointment suggestion, respectively. The appointment suggestion can improve the accuracy of diagnosis and prescription, and improve the efficiency of the medical appointment service.

[0244] The sub-process 1120 can be used to generate a target diagnosis record based on the perception information. The sub-process 1120 can be executed when the inquiry session 830 ends. As shown in FIG. 11B, the sub-process 1120 can include steps 1122, 1124, and 1126. Figure 11

[0245] At step 1122, an initial diagnosis record can be generated based on the perception information.​

[0246] The initial diagnosis record can be an automatically generated diagnosis record. In some embodiments, the initial diagnosis record can include an initial patient history, an initial diagnosis opinion, initial diagnosis prescriptions (e.g., initial treatment prescriptions and initial examination prescriptions), initial medical orders, etc. In some embodiments, key content can be extracted from the perception information based on a diagnosis record template. The key content refers to content related to template fields in the diagnosis record template. The key content can be converted into professional content according to a knowledge dictionary or a term conversion model. Further, the initial diagnosis record is generated by updating the diagnosis record template based on the professional content and a knowledge database. The knowledge database refers to a knowledge database of a registration department, for example, including department visit specifications (e.g., disease description specifications, diagnosis specifications, prescription specifications, medical order specifications, etc.).

[0247] In some embodiments, physical examination data of the patient collected by one or more examination devices in the outpatient process can be obtained, and the initial diagnosis record can be further generated according to the physical examination data. In some embodiments, the initial diagnosis record can be generated by an agent corresponding to the medical outpatient service. The agent can learn the mechanism of generating a diagnosis record from various data (e.g., a diagnosis record template, a knowledge dictionary, a knowledge database, etc.), and process the perception information and the patient data according to the learned mechanism to generate the diagnosis record.

[0248] Step 1124, presenting the initial diagnosis record to the doctor.

[0249] For example, the processing device 210 can control the public terminal to present the initial diagnosis record, for example, when the patient has started the outpatient service. As another example, the processing device 210 can control the doctor terminal to present the initial diagnosis record to the doctor. In some embodiments, the doctor terminal can present the initial diagnosis record to the doctor at a preset time (e.g., after the doctor finishes the daily visit work).

[0250] Step 1126, generating a target diagnosis record based on the initial diagnosis record and feedback information input by the doctor on the initial diagnosis record.

[0251] The feedback information input by the doctor can include modifications and / or confirmations of the initial diagnosis record by the doctor. The target diagnosis record is the diagnosis record after the modifications and / or confirmations by the doctor. In some embodiments, the target diagnosis record can include a target patient history, a target diagnosis opinion, target diagnosis prescriptions (e.g., target treatment prescriptions and target examination prescriptions), target medical orders, etc.

[0252] By generating the target diagnosis record, the manual writing errors of the target diagnosis record can be reduced, and the generation efficiency of the target diagnosis record can be improved. On the other hand, the clerical work of the doctor can be reduced, so that the doctor has more energy to give care to the patient, and the quality of the medical outpatient service is improved.

[0253] Sub-process 1130 can be used to provide a remote accompanying service based on the perception information. The patient can make a request for a remote accompanying service at the pre-consultation. Sub-process 1130 can be performed in the consultation step 830. As shown in FIG. 11B, sub-process 1130 can include steps 1132 and 1134. Figure 11

[0254] Step 1132, based on the perception information, determine whether the patient needs to communicate with the remote accompanying person.

[0255] In some embodiments, the processing device 210 can detect whether the patient makes a request to communicate with the remote accompanying person based on the perception information (e.g., voice data and / or image data). In some embodiments, the processing device 210 can determine the state information of the patient according to the perception information, and determine whether the patient needs to communicate with the remote accompanying person according to the state information of the patient. For example, when the state information indicates that the patient is in a state of high tension, fear, etc., the processing device 210 can determine that the patient needs to communicate with the remote accompanying person.

[0256] When it is determined that the patient needs to communicate with the remote accompanying person, the processing device 210 can perform step 1134.

[0257] Step 1134, at least one terminal device can be controlled to enlarge the second interface element.

[0258] When the patient receives an on-site medical consultation service in the clinic, the processing device 210 can control the public terminal device to enlarge the second interface element. When the patient receives a remote medical consultation service, the processing device 210 can control the patient terminal to enlarge the second interface element. Through the enlarged second interface element, the patient can view the real-time picture of the remote accompanying person and better communicate with the remote accompanying person.

[0259] In some embodiments, when the patient receives an on-site medical consultation service in the clinic, when it is detected that the patient needs to communicate with the remote accompanying person, the processing device 210 can remind the patient to wear an XR device and control the XR device to present image data of the remote accompanying person.

[0260] In some embodiments of the present application, the communication needs of the patient can be detected based on the perception information, and the communication needs can be met in a timely manner, thereby providing more personalized care for the patient and providing a more real and immersive accompanying experience.

[0261] Sub-process 1140 can be used to present medical data to a target user based on perception information. As shown in FIG. 11C, sub-process 1140 can include steps 1142 and 1144. Figure 11

[0262] ​​At step 1142, a control instruction issued by the at least one target user can be obtained based on the perception information, for retrieving at least a portion of the medical data.

[0263] The target user can include at least a patient and a doctor. In some embodiments, the target user further includes a remote companion of the patient. The medical data of the patient can include various data reflecting the health condition of the patient (e.g., electronic medical record, medical image, medical examination result, etc.).

[0264] The control instruction is an instruction for retrieving at least a portion of the medical data (e.g., electronic medical record) for display. For example, the control instruction is for retrieving a three-dimensional model of an organ of interest of the patient in the electronic medical record for display. In some embodiments, the control instruction can also be for setting display parameters (e.g., display angle, display size, display position). In some embodiments, the control instruction can also be for annotating key data on the medical data (e.g., the three-dimensional model of the organ of interest).

[0265] In some embodiments, the perception information can include voice signals collected by a sound sensor, and the control instruction can be obtained by performing semantic analysis on the voice signals. In some embodiments, the target user can issue the control instruction by speaking a preset wake-up word. In some embodiments, the perception information can include optical image data of the target user (e.g., the patient and / or the doctor) collected by an image sensor, and the control instruction can be obtained by performing gesture recognition on the target user in the optical image data. In some embodiments, the target user can issue the control instruction using a control device (e.g., a remote controller, a smart control glove, etc.).

[0266] In some embodiments of the present application, the target user can flexibly adjust the display content and / or the display parameters, for example, by voice, gesture, etc., so as to optimize the user experience and improve the efficiency of medical services.

[0267] At step 1144, in response to the control instruction, at least a portion of the medical data is retrieved and presented on the at least one terminal device.

[0268] For example, the processing device 210 can retrieve at least a portion of the medical data from the storage device, and control the at least one terminal device to present at least a portion of the medical data. When the display parameters are included in the control instruction, the processing device 210 can control the at least one terminal device to present at least a portion of the medical data based on the display parameters.

[0269] In some embodiments of this application, multiple target users can browse medical data together through at least one terminal device, and can synchronously change the presentation content and presentation method of medical data on different terminal devices. This helps to improve the communication efficiency between target users and enhance the interactivity of the medical treatment process.

[0270] Figure 13 This is a schematic diagram of an exemplary hospitalization process 1300 according to some embodiments of this application. Figure 13 As shown, the inpatient process 1300 includes inpatient admission 1310, admission inquiry 1320, inpatient procedure 1330, discharge 1340, follow-up 1350, etc., or any combination thereof. Different user services can be provided to relevant users of the inpatient process 1300 as the patient is at different stages of the inpatient process 1300. Relevant users of the inpatient process can include the patient, healthcare providers (e.g., doctors, nurses, etc.) providing medical services to the patient during the inpatient process, and the patient's visitors. In some embodiments, the processing device 210 (e.g., service module 430, an agent corresponding to the inpatient service / process configured on the processing device 210) can execute the steps involved in multiple stages of the inpatient process 1300.

[0271] At the hospital admission stage 1310, the patient can complete the relevant admission procedures. In some embodiments, hospitalization services related to the hospital admission stage 1310 can be provided to the relevant user. For example, hospitalization services may include guiding the patient to the patient's corresponding hospital ward, explaining the hospitalization rules to the patient, conducting admission examinations for the patient, generating the patient's registration record, etc., or any combination thereof. Further descriptions of hospitalization services can be found elsewhere in this application. For example, see... Figure 14 And its related descriptions.

[0272] In the admission inquiry stage 1320, the patient may receive an admission inquiry, which is used to collect basic information about the patient. In some embodiments, admission inquiry services related to the admission inquiry stage 1320 may be provided to relevant users. For example, admission inquiry services may include performing one or more inquiries on the patient, generating an admission record for the patient, or any combination thereof. The admission inquiry of the patient may be conducted in accordance with... Figure 9 and Figure 10The second inquiry described in the middle is conducted in a similar manner, which is not repeated here. In some embodiments, the processing device 210 can obtain feedback information about the admission record input by the doctor through the doctor terminal. The feedback information can include information that is not present in the admission record but is considered essential by the doctor. Further, the processing device 210 can determine a supplementary inquiry content of a supplementary inquiry according to the feedback information, and cause the public terminal device in the hospital ward to conduct the supplementary inquiry according to the supplementary inquiry content. The supplementary inquiry can be conducted in a similar manner to the admission inquiry. Then, the processing device 210 can obtain supplementary perception information collected by one or more perception devices during the supplementary inquiry, and update the admission record based on the supplementary perception information. Alternatively, the doctor can also directly go to the hospital ward to conduct a supplementary inquiry for the patient. The processing device 210 can obtain supplementary perception information collected by one or more perception devices during the supplementary inquiry. The processing device 210 can further update the admission record based on the supplementary perception information.

[0273] In the hospitalization link 1330, the patient can stay in the hospital (e.g., hospital ward) for a period of time to receive around-the-clock medical care. In some embodiments, a hospital ward service related to the hospitalization link 1330 can be provided to relevant users. For example, the hospital ward service can include nursing service 1332, ward round service 1334, visiting service 1336, and the like, or any combination thereof.

[0274] The nursing service 1332 is used to provide direct nursing to the patient, including medication, physical examination, monitoring of vital signs, and assistance with activities of daily living. More description about the nursing service can be found elsewhere in this application. For example, see Figure 15 and related description thereof.

[0275] The ward round service 1334 involves a ward round conducted by a medical team (e.g., at least one doctor) of the patient in the hospital ward, where the medical team can review and discuss the patient's condition and care plan in the ward round. For example, the ward round service 1334 can include presenting data for facilitating communication between the doctor and the patient, generating a ward round record, presenting a virtual ward space to one or more remote doctors, and the like, or any combination thereof.

[0276] By way of example only, the processing device 210 can obtain sensory information collected by at least one physician when conducting a ward round in a hospital ward and / or physical examination data of a patient collected by one or more examination devices during the ward round, and generate a ward round record based on the sensory information and / or the physical examination data of the patient. The ward round record is used to record relevant data of the ward round, including ward round time, participants, patient data, communication content between the patient and the at least one physician, medical orders at the time of the ward round, and the like. By way of another example, the processing device 210 can generate a virtual ward space based on the sensory information, and present the virtual ward space to one or more remote physicians through one or more XR devices of the one or more remote physicians. The virtual ward space refers to a digitalized environment of the ward round in the ward. Optionally, the remote physicians can communicate with at least one physician and a patient in the hospital ward through the XR devices.

[0277] The visitation service 1336 allows remote visitors to remotely communicate with a patient. For example, the visitation service 1336 can include generating a virtual visitation space for the patient and the remote visitor, presenting the virtual visitation space to the patient and the remote visitor, or any combination thereof.

[0278] By way of example only, in response to an access request, the processing device 210 can obtain first current information of the patient and second current information of the remote visitor, and generate a virtual access space for the patient and the remote visitor based on the first current information and the second current information. The first current information can indicate a current state and / or a current environment of the patient. The second current information can indicate a current state and / or a current environment of the remote visitor. The virtual visitation space refers to a digital environment presented to the patient and the remote visitor during the visitation. Further, the processing device 210 can present the virtual access space to the patient and the remote visitor through a public terminal device in the hospital ward and a remote terminal device of the remote visitor, respectively.

[0279] At the discharge stage, the patient can handle the discharge procedure. In some embodiments, a relevant user can be provided with discharge services related to the discharge stage 1340 to guide the patient to discharge from the hospital. For example only, in response to a discharge instruction received from a doctor terminal of a doctor of the patient, the processing device 210 can obtain a target hospitalization record of the patient. The target hospitalization record can record information of the patient's hospitalization process, such as a medical history, treatments received, prescribed drugs, test results, and a discharge summary. In addition, the processing device 210 can generate discharge data according to the target hospitalization record, and present the discharge data to the patient through a public terminal device in the hospital ward. The discharge data can include a discharge summary, doctor's orders on discharge, guidance information on discharge procedures, guidance information on discharge procedures, discharge fees, payment methods, and the like, or any combination thereof. In response to determining that the patient performs the discharge operation, the processing device 210 can generate a discharge record corresponding to the patient. The discharge record is used to record relevant data of the discharge, including discharge time, discharge summary, doctor's orders on discharge, discharge fees, payment methods, patient's state when discharging, and the like.

[0280] In the follow-up stage 1350, the patient can be provided with continuous care after discharging to ensure continuous rehabilitation, handle any remaining health problems, and prevent readmission. In some embodiments, a relevant user can be provided with follow-up services related to the follow-up stage 1350. For example only, the processing device 210 can determine a follow-up plan of the patient based on the target hospitalization record of the patient. The follow-up plan is used to guide how to provide follow-up services for the patient. In some embodiments, the follow-up plan can include one or more follow-ups performed at one or more scheduled times. In addition, the processing device 210 can cause the doctor terminal of the doctor and the patient terminal of the patient to remind the doctor and the patient, respectively, according to the follow-up plan. The follow-up can be performed offline or remotely in a virtual follow-up space.

[0281] In some embodiments, after the follow-up is performed, the processing device 210 can generate a follow-up record corresponding to the patient. The follow-up record is used to record data related to the follow-up, including a time corresponding to the follow-up, updated orders corresponding to the follow-up, health monitoring information corresponding to the follow-up, and the like. In some embodiments, the processing device 210 can update the follow-up plan. For example, the processing device 210 can obtain health monitoring information of the patient, and update the follow-up plan based on the health monitoring information. The health monitoring information can be collected by one or more home monitoring devices in the patient's home.

[0282] In some embodiments, the processing device 210 can monitor data sources that collect data related to the patient’s inpatient procedure. In response to detecting a data update in at least one of the data sources, the processing device 210 can perform EOI detection based on the updated data collected by the at least one data source. If an EOI occurs, the processing device 210 can perform one or more preset operations corresponding to the EOI to provide at least a portion of the inpatient service.

[0283] The updated data refers to data collected by the at least one data source that has not yet been processed by the processing device 210. For example, if the at least one data source includes a sensing device, the updated data can include sensing information collected by the sensing device, such as image data collected by an image sensor, sound data collected by a sound sensor, etc. As another example, if the at least one data source includes a terminal device of a doctor associated with the patient, the updated data can include input data of the patient input by the doctor through the terminal device. As another example, if the at least one data source includes a vital sign monitor, the updated data can include vital signs of the patient. As yet another example, if the at least one data source includes a medical examination department, the updated data can include examination results of the patient.

[0284] The EOI refers to a specific event or action that requires attention. The EOI detection refers to processing the updated data collected by the at least one data source to detect whether one or more EOIs have occurred. For example, exemplary EOIs can include: the patient checking into a hospital room, conducting an admission examination on the patient, at least one doctor visiting the hospital room, performing a nursing operation or a medical examination on the patient, the patient initiating a service request, obtaining or updating a doctor’s order for the patient, an abnormal physiological state of the patient, the doctor issuing an instruction to the patient, etc., or the like, or any combination thereof.

[0285] In some embodiments, the at least one data source can include multiple data sources that collect data related to the same EOI. In some embodiments, the at least one data source can include data sources that collect data related to multiple EOIs.

[0286] In some embodiments, the processing device 210 can perform EOI detection based on EOI detection rules. The EOI detection rules refer to rules that need to be involved when performing EOI detection. In some embodiments, the EOI detection rules can be determined based on historical records of EOI detection, or manually set by a user (e.g., a doctor, a nurse, a technician, etc.).

[0287] In some embodiments, each link of the inpatient procedure can correspond to one or more EOIs, and different links of the inpatient procedure can correspond to different types of EOIs. Therefore, the processing device 210 can perform EOI detection on the updated data based on the current link of the patient in the inpatient procedure.

[0288] The preset operations can include general operations and / or specific operations. The general operations refer to operations that need to be performed as long as an EOI occurs, regardless of the type of the EOI, for example. For example, the general operations can include generating a record related to the EOI, sending the record related to the EOI to a relevant user for confirmation or to a storage device for storage, and the like.

[0289] The specific operations refer to operations that are performed when a specific type of EOI occurs. For example, updating a daily plan of a patient according to update data can be determined as a specific operation corresponding to an EOI of updating a medical order of the patient by a doctor. As another example, providing a notification related to the EOI to a medical service provider can be determined as a specific operation corresponding to an EOI of an abnormal physiological state of a patient.

[0290] For example, when the EOI includes that the patient is admitted to a hospital ward, one or more preset operations can include: determining, according to patient data of the patient, an inquiry content of an inquiry to be made to the patient after the patient is admitted to the hospital ward, causing a terminal device in the hospital ward to make the inquiry based on the inquiry content; and obtaining sensing information collected by one or more sensing devices in the hospital ward during the inquiry, and generating an inpatient record for the patient based on the sensing information.

[0291] As another example, when the EOI includes obtaining an inpatient guidance request, one or more preset operations can include obtaining a first location of a terminal of the patient and a second location of a hospital ward of the patient, determining a planned path from the first location to the second location according to a real-time map of the hospital, and causing the terminal device of the patient to present guidance information related to the planned path to the patient.

[0292] As another example, when the EOI includes that a condition for performing an admission examination is met, one or more preset operations can include controlling an intelligent nursing cart to guide a nurse to the hospital ward to perform an admission examination on the patient.

[0293] As another example, when the EOI includes obtaining or updating a medical order of the patient, one or more preset operations can include determining, according to patient data of the patient and the medical order or the updated medical order, a daily plan of the patient every day when the patient is hospitalized, presenting the daily plan to the patient through a terminal device in the hospital ward of the patient, and presenting the daily plan to a corresponding nurse of the patient through a terminal device of the nurse. The daily plan can include at least one medical procedure that needs to be performed on the patient every day.

[0294] As another example, when the EOI includes that at least one doctor performs a ward round in a hospital ward of the patient, one or more preset operations can include obtaining sensing information collected by one or more sensing devices in the hospital ward when the at least one doctor performs the ward round in the hospital ward, and generating a ward round record based on the sensing information.

[0295] As another example, when the EOI includes an obtain visit request, the one or more preset operations can include obtaining first current information of the patient and second current information of the remote visitor, generating a virtual visit space for the patient and the remote visitor based on the first current information and the second current information, and presenting the virtual visit space to the patient and the remote visitor respectively.

[0296] As another example, when the EOI includes a received discharge instruction, the one or more preset operations can include obtaining a target hospitalization record of the patient, generating discharge data based on the target hospitalization record, and presenting the discharge data to the patient through a terminal device in a patient ward.

[0297] As another example, when the EOI includes that the patient has been discharged, the one or more preset operations can include determining a follow-up plan of the patient according to a target hospitalization record of the patient. The follow-up plan can include one or more follow-ups performed at one or more scheduled times. For each of the one or more follow-ups, the one or more preset operations can further include causing a terminal device of the attending physician and a terminal device of the patient to remind the attending physician and the patient respectively according to the scheduled time of the follow-up.

[0298] In some embodiments, the one or more preset operations can be performed based on a data source corresponding to the EOI. For example, when the at least one data source includes a plurality of data sources that collect data related to the same EOI, the one or more preset operations can be performed based on a combination of the data related to the same EOI collected by the plurality of data sources. As another example, the at least one data source includes a data source that collects data related to a plurality of EOIs, and the one or more preset operations corresponding to at least two EOIs of the plurality of EOIs can be different.

[0299] In some embodiments, the processing device 210 can be configured with an agent, and the agent can perform at least part of the flow 1300. For example, the agent can learn an EOI detection rule from historical records and perform EOI detection according to the EOI detection rule. As another example, the agent can learn a correspondence between an EOI and a preset operation from historical records and determine one or more preset operations corresponding to the EOI according to the correspondence.

[0300] In some embodiments, the agent can further learn the EOI detection rule and / or the correspondence between the EOI and the preset operation based on patient data of different patients.

[0301] Figure 14is a flowchart of an example procedure 1400 for providing inpatient services, according to some embodiments of the present application. The procedure 1400 can be performed when the processing device 210 detects that a patient has been admitted (e.g., the patient has completed the relevant procedures for admission).

[0302] At step 1410, the processing device 210 can direct the patient to a hospital room.

[0303] For example, the processing device 210 can instruct the patient terminal of the patient to direct the patient to the hospital room. By way of example only, in response to the inpatient direction request, the processing device 210 can obtain a first location of the patient terminal of the patient and a second location of the hospital room, and determine a planned path from the first location to the second location according to a real-time map of the hospital. Then, the processing device 210 can instruct the patient terminal to present the patient with direction information related to the planned path.

[0304] At step 1420, the processing device 210 can provide inpatient education to the patient.

[0305] The inpatient education can be used to introduce the patient to inpatient information (e.g., inpatient procedures, inpatient operations, pre-admission costs, payment methods, etc.), inpatient hospital rules, the hospital environment, the patient’s doctors and / or nurses, etc. In some embodiments, the processing device 210 can cause the patient terminal to present a third virtual character providing the inpatient education.

[0306] At step 1430, the processing device 210 can assist a nurse with inpatient preparation.

[0307] The inpatient preparation can be performed by the nurse to prepare the hospital supplies for the patient. In some embodiments, the processing device 210 can present an inpatient preparation notification to the nurse through a nurse terminal 1405 within a nurse station or the smart nursing cart 240-4 to assist the nurse with the inpatient preparation. The inpatient preparation notification can include patient data of the patient, a list of hospital supplies for the patient, room information of the patient, information for conducting inpatient examinations on the patient, etc.

[0308] The inpatient examination can also be referred to as an admission examination, which is conducted after the patient is admitted into the hospital room. The inpatient examination can be used to collect information about the current medical condition of the patient (e.g., vital signs, basic health data, etc.). The inpatient examination can include checking blood pressure, blood sugar, heart rate, body temperature, etc., or any combination thereof.

[0309] At step 1440, the processing device 210 can issue a reminder to perform the inpatient examination. The reminder can include a message reminder, a sound reminder, a pop-up reminder, etc. For example, the processing device 210 can instruct the nursing terminal device 1405 or the smart nursing cart 240-4 to present the reminder.

[0310] In some embodiments, the processing device 210 can determine whether the patient meets a condition for conducting an admission examination in the hospital room. The condition for conducting an admission examination in the hospital room can include that the patient has arrived at the hospital room for a period of time. If the patient meets the condition, the processing device 210 can issue a reminder to conduct an admission examination.

[0311] At step 1450, the processing device 210 can guide the nurse to the hospital room. In some embodiments, the processing device 210 can control the motion of the smart cart 240-4 to guide the nurse into the hospital room.

[0312] At step 1460, the patient can be subjected to an admission examination.

[0313] For example, after the nurse arrives at the hospital room, the patient can be subjected to an admission examination using one or more examination devices to collect physical examination data of the patient. In some embodiments, the processing device 210 can instruct the smart cart to present information related to the admission examination to the nurse during the admission examination. For example, the smart cart can display an admission examination illustration, an electronic medical record of the patient, etc.

[0314] At step 1470, the processing device 210 can generate an admission record.

[0315] The admission record refers to a record indicating that the patient has been admitted to the hospital room and / or a status of the patient when admitted to the hospital room. The admission record can include admission information (e.g., an admission number, clinical information, an admission time, a pre-admission amount of hospitalization fees, a payment method, etc.), physical examination data collected at the time of the first diagnosis, etc.

[0316] In some embodiments, the processing device 210 can generate the admission record based on an admission record template and the physical examination data. In some embodiments, the processing device 210 can further generate the admission record based on the electronic medical record of the patient. In some embodiments, the processing device 210 can present the admission record to the nurse through the smart cart 240-4 or the nurse terminal 1405, and generate a target admission record based on the admission record and feedback information of the admission record input by the nurse through the smart cart 240-4 or the nurse terminal 1405. The feedback information can include confirmation instructions, modification instructions, etc. input by the nurse.

[0317] In some embodiments, the processing device 210 can be configured with an agent (e.g., an admission agent) that can participate in performing one or more steps of the process 1400. For example, the agent can guide the patient into the hospital room, provide admission education to the patient, assist the nurse in conducting an admission examination, generate an admission record, etc.

[0318] ​​​​​​​​According to some embodiments of the present application, a patient can be provided with inpatient services in a semi-automatic manner with the help of a medical service system (e.g., the smart nursing cart 240-4) and / or an intelligent agent, so that the labor cost can be reduced and the efficiency of inpatient services can be improved.

[0319] Figure 15 FIG. 15 is a schematic diagram of an exemplary process 1500 for providing nursing services, according to some embodiments of the present disclosure. In some embodiments, the process 1500 can be performed every day during a patient’s hospitalization, so as to provide nursing services for the patient.

[0320] At step 1502, the processing device 210 can determine a daily plan for the patient according to the patient data of the patient and the doctor’s orders for the patient.

[0321] The doctor’s orders for the patient refer to the instructions or commands given by the doctor to the patient. In some embodiments, the doctor’s orders for the patient can be stored in a storage device, and can be updated when any doctor issues new orders for the patient. The processing device 210 can obtain the latest version of the orders from the storage device. In some embodiments, the processing device 210 can monitor various hardware devices to detect whether the doctor’s orders for the patient are updated. For example, when the patient is provided with an admission inquiry service and / or a ward round service, the doctor of the patient can issue new orders to the patient. The processing device 210 can detect the new orders based on the perception information collected by the perception device during the admission inquiry service and / or the ward round service. Once the new orders are detected, the new orders can be stored in the storage device. As another example, the doctor can update the patient’s orders stored in the storage device through a doctor terminal. In some embodiments, the processing device 210 can determine the doctor’s orders according to the electronic medical record of the patient.

[0322] In some embodiments, the processing device 210 can determine a daily plan for the patient according to the patient data of the patient and the doctor’s orders for the patient. The daily plan can include at least one medical operation that needs to be performed on the patient on the same day. Exemplary medical operations can include nursing operations, examination operations, etc.

[0323] At step 1504, the processing device 210 can present the daily plan to the patient through a public terminal device (e.g., a bedside terminal) in the hospital ward.

[0324] At step 1506, the processing device 210 can present the daily plan to the nurse corresponding to the patient through a nurse terminal (e.g., a terminal device in a nurse workstation, etc.).

[0325] At step 1508, when the daily plan includes at least one nursing operation, the nurse can perform at least one nursing operation on the patient, and the processing device 210 can assist the nurse in performing at least one nursing operation according to the daily plan.

[0326] As Figure 15 indicated, for each of the at least one nursing operation, the processing device 210 can control the smart nursing cart to guide the nurse to the hospital room for the nursing operation according to the scheduled time of the nursing operation. For example, before the scheduled time of the nursing operation, the smart nursing cart can be controlled to move to the nurse station to notify the nurse that the nursing operation needs to be performed on the patient. Then, the smart nursing cart can be controlled to move and guide the nurse to the hospital room of the patient. The processing device 210 can further control the smart nursing cart to present the nursing instructions related to the nursing operation after the nurse arrives at the hospital room.

[0327] At step 1510, the processing device 210 can generate a nursing record.

[0328] The nursing record refers to the record of the nursing operation applied to the patient and / or the state of the patient (e.g., vital signs and other physiological measurements) before, after, or during the nursing operation. In some embodiments, the processing device 210 can obtain the fifth perception information collected by one or more perception devices in the hospital room while performing the at least one nursing operation, and generate the nursing record based on the fifth perception information. In some embodiments, the nursing record can be displayed to the nurse for confirmation through the smart nursing cart or the nurse terminal.

[0329] In some embodiments, the processing device 210 can be configured with an agent (e.g., a nursing agent), and the agent can participate in the execution of the process 1500. For example, the agent can determine the daily plan for the patient, present the daily plan to the patient and / or the nurse, assist the nurse in performing the at least one nursing operation, and generate the nursing record. In some embodiments, the processing device 210 configured with the nursing agent can be integrated into the hospital bed, or into a public terminal device in the hospital room, or into the smart nursing cart.

[0330] According to some embodiments of the present application, the automatic generation of the daily plan and the nursing record can significantly reduce the workload of the nurse. This automation enables the nurse to focus more on direct patient care rather than administrative tasks. In addition, the monitoring of the medical order updates ensures the timely update of the daily plan. This proactive approach improves nursing efficiency and quality of care by ensuring timely adjustments to interventions and care plans according to the latest medical instructions.

[0331] Figure 16 is an exemplary schematic diagram of a process 1600 for surgical planning and execution according to some embodiments of the present application. The process 1600 can be executed by the processing device 210 (e.g., a service module 430, an agent configured on the processing device 210 corresponding to a surgical service / process).

[0332] At step 1610, a surgical plan is made.

[0333] The surgical plan refers to a process of making a surgical plan (e.g., an optimal surgical plan or multiple feasible surgical plans) for a patient. In some embodiments, the processing device 210 can generate a surgical plan based on patient data (e.g., patient personal data, historical diagnosis and treatment data, medical examination data, etc.), physician feedback information (e.g., second feedback information), and / or perception information (e.g., fourth perception information). In some embodiments, the processing device 210 can generate an initial surgical plan according to the patient data, present the patient data to the physician through the physician terminal, and generate a surgical plan according to the initial surgical plan and the second feedback information about the initial surgical plan input by the physician through the physician terminal.

[0334] In some embodiments, the processing device 210 can determine a surgical difficulty coefficient according to the patient data, and further determine whether to hold an expert meeting according to the surgical difficulty coefficient. In response to determining that an expert meeting is needed, the processing device 210 presents a virtual meeting space through the physician terminal of the physician and the remote terminal device (e.g., an XR device) of the remote expert, respectively. During the expert meeting, the processing device 210 obtains fourth perception information collected by the physician terminal and the remote terminal device. The processing device 210 generates a surgical plan based on the patient data and the fourth perception information.

[0335] In some embodiments, the processing device 210 can generate a risk assessment result of the surgical plan by processing the surgical plan and at least part of the patient data using a risk assessment model. The processing device 210 determines a risk prevention measure according to the risk assessment result, and presents the risk assessment result of the surgical plan and the risk prevention measure to the physician. In some embodiments, the processing device 210 can generate a surgical plan by performing steps 1710-1740 in Figure 17

[0336] Step 1620, performing a surgical simulation.

[0337] The surgical simulation refers to a process in which a physician performs a surgical practice in a safe and controllable environment in order to improve the surgical plan and / or improve the surgical skills of the physician. For example, for a complex surgery or a rare surgery, the physician can perform a simulated surgery on a virtual patient in a virtual surgery scene (e.g., an extended reality surgery scene) using an XR device according to the surgical plan. Identify potential risk points in the surgical process and develop corresponding risk prevention measures. In addition, for multiple surgical plans, the physician can simulate each surgical plan in a virtual surgery scene through an extended reality device in order to compare the advantages and disadvantages of different surgical plans and determine the optimal surgical plan.

[0338] ​In some embodiments, the processing device 210 can generate a virtual surgical scene for surgical simulation based on the surgical plan, and present the virtual surgical scene to the doctor through the doctor's terminal. The processing device 210 can obtain interactive instructions related to the virtual surgical device input by the doctor through the doctor's terminal or an interactive device corresponding to the virtual surgical device. The processing device 210 can update the virtual surgical site and virtual surgical device in the virtual surgical scene according to the interactive instructions.

[0339] In some embodiments, the processing device 210 can determine potential emergency situations in the virtual surgical scenario based on interactive instructions. Based on the potential emergency situation, the processing device 210 can update the virtual surgical site and virtual surgical equipment in the virtual surgical scenario. As an example only, the processing device 210 can obtain simulation data of the virtual surgical site and virtual surgical equipment during the surgical simulation process. The processing device 210 can determine whether the surgical plan needs optimization based on the simulation data. In response to determining that the surgical plan needs optimization, the processing device 210 can update the surgical plan based on the simulation data. In some embodiments, step 1620 can be omitted.

[0340] Step 1630: Prepare the patient for surgery.

[0341] Preoperative patient preparation refers to the preparations made by the patient before the surgical procedure or before entering the operating room for the surgical procedure. For example... Figure 16 As shown, preoperative patient preparation may include preoperative education (or preoperative care) 1632 and preoperative instruction 1634.

[0342] Preoperative education refers to the process of explaining the patient's condition and surgical plan to the patient and / or their family, and predicting the patient's postoperative recovery. Preoperative education can be achieved through... Figure 17 The process is as follows: 1700. Preoperative guidance refers to the process by which patients complete preoperative preparations before surgery. Preoperative education can be conducted according to... Figure 18 The process is carried out at 1800.

[0343] Step 1640, surgery performed.

[0344] Surgery execution refers to operations performed on the patient in the operating room in relation to the surgery. In some embodiments, the processing device 210 can obtain first perception information collected by one or more first perception devices in the operating room during the surgery of the patient and perform EOI detection on the first perception information. In response to determining that an EOI has occurred, the processing device 210 can perform one or more preset operations corresponding to the EOI. For example, the EOI can include a target surgical tool instruction issued by a surgical participant, and the one or more preset operations corresponding to the EOI can include causing the intelligent nurse robot to deliver the target surgical tool to the surgical participant. As another example, the first perception information can include an image of a surgical tool captured by an image sensor, the EOI can include a count of the surgical tool being less than a preset value, and the one or more preset operations corresponding to the EOI can include controlling the intelligent nurse robot to replenish the surgical tool. As another example, the EOI can include detecting a surgical risk, and the one or more preset operations corresponding to the EOI can include providing a notification about the surgical risk. As another example, the EOI can include that the surgery has been completed, and the one or more preset operations corresponding to the EOI can include generating a surgery record based on the first perception information.

[0345] At step 1650, post-surgery review is performed.

[0346] The post-surgery review can include at least one of updating a medical recommendation report, generating a surgery result and a doctor’s operation record (to facilitate review of the surgery process), and developing a post-surgery care plan.

[0347] Figure 17 FIG. 17 is a schematic diagram of an example procedure 1700 of pre-surgery education according to some embodiments of the present disclosure.

[0348] At step 1710, explanation materials for explaining the surgery plan are generated.

[0349] The explanation materials are configured to explain information related to the surgery plan, such as an explanation note of the surgery plan, an execution process of the surgery plan at the surgery site of the patient, a post-surgery rehabilitation process of the patient using the surgery plan, etc. In some embodiments, the explanation materials can include text, image, audio, or video. In some embodiments, there can be multiple surgery plans. The processing device 210 can generate explanation materials corresponding to each surgery plan.

[0350] In some embodiments, the explanation materials are generated based on a digital twin model (e.g., a three-dimensional anatomical model) of the surgery site of the patient. For example, the processing device 210 can simulate a surgery process and result (e.g., a post-surgery incision size) based on the surgery plan on the three-dimensional anatomical model of the surgery site of the patient.

[0351] In some embodiments, the explanatory materials include surgical videos. The surgical videos demonstrate the surgical procedure at the patient's surgical site. For example, in a surgical plan where invasive surgery is the type of procedure, the surgical video shows the appearance of the patient's surgical site before the incision, the incision process with a scalpel, the removal of the lesion, the suturing process, and the appearance after the suturing process.

[0352] In some embodiments, explanatory materials may include a patient's postoperative recovery process. For example, surgical videos may further demonstrate the patient's postoperative recovery process. Processing device 210 may predict the patient's postoperative recovery process based on patient data. The recovery process reflects the healing of the patient's vital signs and surgical wound. In some embodiments, surgical videos may further demonstrate the risks the patient faces during or after surgery (e.g., intraoperative and postoperative risks). Postoperative risks refer to adverse conditions the patient may face after surgery.

[0353] Step 1720: The explanatory materials can be presented to both the patient and the doctor simultaneously via at least one terminal device.

[0354] In some embodiments, at least one terminal device includes a patient's patient terminal, a doctor's doctor terminal, and a home terminal device. For example... Figure 17 As shown, the processing device 210 can simultaneously present explanatory materials to the patient, doctor, and family members via the XR device 260-2 worn by the patient, the XR device 270-2 worn by the doctor, and the XR device 1722 worn by the family member. In some embodiments, the processing device 210 can simultaneously present explanatory materials via at least one terminal device, with the doctor providing explanations. Users can choose to update the displayed content and display method of the explanatory materials by inputting commands through their own terminal devices.

[0355] Step 1730: Based on the second sensing information collected by one or more second sensing devices during the explanation of the surgical plan, determine the first feedback information about the surgical plan.

[0356] A second sensing device refers to a sensing device configured to receive input information (e.g., text information, voice information, etc.) from a user (e.g., a doctor or patient). For example, a second sensing device may be a sound sensor (e.g., a microphone), an image sensor, etc., integrated into a user terminal device.

[0357] The first feedback information includes feedback from preoperative education participants (e.g., doctors, patients, and patients' families) regarding the surgical plan. For example, the first feedback information may include at least selection information for the surgical plan. Selection information refers to the result of selecting a surgical plan. In some embodiments, the first feedback information may also include modification information for the selected surgical plan. The processing device 210 may analyze and process the second perceptual information using speech recognition, image recognition, or the like to determine the first feedback information.

[0358] Step 1740: Confirm or update the surgical plan based on the first feedback information.

[0359] In some embodiments, the processing device 210 can determine a target surgical plan from multiple surgical plans based on first feedback information. In some embodiments, the processing device 210 can update the patient's surgical plan based on the first feedback information. In some embodiments, after determining the target surgical plan from multiple surgical plans, the processing device 210 can further obtain a first confirmation instruction and a second confirmation instruction. The first confirmation instruction is an instruction regarding the surgical plan entered by the patient through a patient terminal. The second confirmation instruction is an instruction regarding the surgical plan entered by the patient's family member through a home terminal device. In response to receiving the first and second confirmation instructions, the processing device 210 can cause the patient terminal, doctor terminal, and home terminal device to respectively present operating protocols. The processing device 210 can obtain signature information of the surgical protocol from the patient terminal, doctor terminal, and home terminal device respectively.

[0360] The input method for confirmation instructions or signature information may include button input, gesture input, voice input, etc. In some embodiments, the input method for confirmation instructions or signature information may be fingerprint input. The processing device 210 may verify the user's identity based on the fingerprint input by the user, for example, using blockchain verification technology.

[0361] Step 1750: Generate a video explaining the surgical plan.

[0362] In some embodiments, such as Figure 17 As shown, processing device 210 can generate explanatory annotations for the surgical plan based on the second sensory information. Processing device 210 can create an explanatory video of the surgical plan based on the surgical video and the explanatory notes. The explanatory notes may include the surgeon's explanations of one or more frames in the surgical video and the corresponding operations. For example, the explanatory notes may include explanations of incision site selection, incision path, incision length, etc. The explanatory notes can be incorporated into the surgical video in text or audio form to obtain the explanatory video.

[0363] In some embodiments, the processing device 210 may acquire interactive instructions related to the explanatory materials from at least one terminal device, and update the explanatory materials displayed through the at least one terminal device based on the interactive instructions. Interactive instructions refer to control or modification instructions for the explanatory materials input by a patient through a patient terminal or by a doctor through a doctor terminal. For example, if the explanatory material is a surgical video, the interactive instruction might be "rewind the video 15 seconds."

[0364] According to some embodiments of this application, in a virtual patient education space, the patient's condition progression and surgical plan can be explained to the patient and their family via video, enabling remote doctor-patient communication. This helps the patient and their family to quickly understand the situation, improves the efficiency of doctor-patient communication, helps the patient and their family to fully understand the patient's current condition and postoperative risks, reduces the patient's fear and anxiety, and increases the success rate of the surgical procedure.

[0365] Figure 18 This is a schematic diagram of an exemplary preoperative guidance process according to some embodiments of this application.

[0366] Preoperative guidance may include patient transport, patient verification, preoperative education, preoperative cleaning, and establishment of intravenous access. Patient transport refers to moving the patient from his / her current location to the waiting area of ​​the operating room.

[0367] Patient verification refers to verifying whether a patient meets the surgical criteria. For example, patient verification may include: verifying whether the verification subject's identity information matches the target patient for the current surgical procedure; verifying whether the verification subject's surgical schedule is currently available; and verifying whether the verification subject's current physical condition meets the requirements of the surgical procedure. It is understandable that if the verification subject does not meet any of the surgical criteria, the patient's surgical schedule may be postponed or delayed.

[0368] In some embodiments, the processing device 210 may collect the patient's biometric information through one or more third sensing devices in the waiting area and verify the patient's identity based on the biometric information. For example, such as Figure 18 As shown, after transporting the patient to waiting area 1810, processing device 210 can collect the patient's biometric information through one or more third sensing devices 1811 (e.g., image capture device, microphone, fingerprint sensor, etc.) in waiting area 1810, and verify the patient's identity based on the biometric information (e.g., perform patient authentication). In some embodiments, processing device 210 can utilize a nurse agent to verify the patient's identity. For example, the nurse agent can verify the collected biometric information or verify the patient's identity through voice interaction with the patient (e.g., asking the patient's age, name, gender, etc.).

[0369] Preoperative care includes preoperative calming and preoperative education. Preoperative calming refers to preoperative preparation to help reduce the patient's negative emotions (e.g., anxiety, tension, fear, etc.) through language communication, videos, music, etc. Preoperative education refers to preoperative preparation to help the patient understand the surgical procedure. Preoperative cleaning refers to preoperative preparation, such as body cleaning, hair removal (e.g., hair, body hair, etc.), the patient wearing a surgical gown, etc. The establishment of intravenous access refers to the establishment of a route for intravenous injection of drugs in the patient's body to ensure that drugs can be effectively administered to the patient's body during the operation.

[0370] In some embodiments, the processing device 210 can determine a planned path from the current location of the patient to the waiting area and control the smart wheelchair to transport the patient to the waiting area along the planned path. For example, as shown in FIG. 18A, before preoperative operation is performed on the patient according to the surgical plan, the processing device 210 can determine a planned path from the current location 1803 (e.g., a hospital room) of the patient 261 to the waiting area 1810. The processing device 210 can control the smart wheelchair 240-5 to transport the patient 261 from the hospital room 1803 to the waiting area 1810 along the planned path. Figure 18

[0371] In some embodiments, the processing device 210 can determine a planned path from the current location to the waiting area based on a hospital map. In some embodiments, the processing device 210 can configure a nurse agent that performs certain tasks on behalf of a nurse and can present a virtual nurse character. The processing device 210 can use the nurse agent to control the smart wheelchair to transport the patient from the current location to the waiting area. In some embodiments, after transporting the patient to the waiting area, the processing device 210 can perform patient verification.

[0372] In some embodiments, the processing device 210 can determine preoperative care materials for the patient according to the patient data and the surgical plan. During the transportation of the patient to the waiting area, the processing device 210 can use the patient terminal to perform preoperative education on the patient according to the preoperative care materials. The preoperative care materials can include videos, music, images, text, and other materials related to surgical explanation and / or emotional relaxation.

[0373] In some embodiments, the processing device 210 can use the nurse agent to provide preoperative education to the patient. For example, as shown in FIG. 18B, the processing device 210 can use the nurse agent to provide preoperative education to the patient 261 during the transportation of the patient 261 to the waiting area 1810. The nurse agent can use the patient terminal to play a video to the patient 261 to help the patient 261 understand the surgical procedure and help the patient 261 relax. Figure 18 ​As shown, the processing device 210 can present the virtual nurse figure 1823 on the XR device 260-2 worn by the patient 261, and the virtual nurse figure 1823 can explain the preoperative care materials to the patient 261. In some embodiments, the virtual nurse figure 1823 can have voice interaction with the patient 261 to alleviate the negative emotions of the patient or answer the questions of the patient through communication. In some embodiments, the processing device 210 can determine whether it is necessary to alleviate the emotions of the patient by collecting the facial expressions, physiological features, tone of voice, etc. of the patient.

[0374] In some embodiments, the processing device 210 can use the nurse agent to guide the nurses to perform the preoperative cleaning and / or the venous access establishment.

[0375] In some embodiments, in the process of transporting the patient to the waiting area, the processing device 210 can obtain third perception information related to a part of the planned path from the current position of the smart wheelchair to the waiting area (for example, the part of the planned path that the smart wheelchair has not traveled) through one or more fourth perception devices in the hospital. Based on the third perception information, the processing device 210 can determine the potential risks of the untraveled part of the planned path, and update the untraveled part based on the potential risks.

[0376] The one or more fourth perception devices can include image capture devices (for example, infrared monitoring cameras 1813), laser radars, etc. The one or more fourth perception devices can be installed at positions such as the smart wheelchair, the hospital ceiling, or the hospital wall, etc.

[0377] In some embodiments of the present application, through the above-mentioned preoperative guidance process, a humanized, transparent, and efficient preoperative preparation process can be provided, and the preoperative preparation items can be dynamically adjusted according to the patient feedback, thereby improving the efficiency of the preoperative preparation. Through the above-mentioned process of transporting the patient and verifying the identity of the patient, human errors are avoided, and the safety of the entire surgical process is improved. Using the virtual nurse image to assist in completing many preoperative preparation tasks can save labor costs.

[0378] Figure 19 is a schematic diagram of an exemplary surgical execution process 1900 according to some embodiments of the present application. The surgical execution process can include preoperative preparation, intraoperative matters, and postoperative matters. As shown, Figure 19 As shown, the preoperative preparation (i.e., the steps before the surgical execution) can include steps 1911, 1913, and 1915.

[0379] Step 1911, activate the operating room.

[0380] Activating the operating room can include opening the operating room door, activating surgical equipment in the operating room, monitoring equipment, adjusting parameters in the operating room, verifying surgical equipment status, etc. In some embodiments, the processing device 210 can control the intelligent robotic nurse to activate the operating room or guide the nurse to activate the operating room. For example, the processing device 210 can control the intelligent robotic nurse to automatically activate the operating room equipment at a predetermined surgery time, and adjust the room temperature, humidity and air quality.

[0381] Step 1913, prepare surgical tools.

[0382] The surgical tools can include surgical instruments and surgical consumables. In some embodiments, the processing device 210 can control the intelligent robotic nurse to prepare the surgical tools in the operating room before the surgery according to the surgical plan. In some embodiments, the processing device 210 can further control the intelligent robotic nurse to sterilize the operating table and arrange the operating table (for example, arrange the positions of various surgical tools on the operating table).

[0383] Step 1915, confirm and / or anesthetize the patient. Patient confirmation refers to the identity of the patient being confirmed. The patient is anesthetized refers to the patient being anesthetized.

[0384] Step 1920, execute the surgical procedure. In some embodiments, as shown in Figure 19 , the matters during the surgical procedure (for example, intraoperative matters) can include remote collaboration, tool transfer, image interaction, intraoperative planning and navigation, and real-time alerts.

[0385] Remote collaboration refers to remotely participating in and / or guiding the surgical procedure.

[0386] Tool delivery refers to delivering surgical tools to surgical performers during the surgical procedure. In some embodiments, the processing device 210 can identify instructions issued by surgical participants for target surgical tools according to first perception information collected by one or more first perception devices in the operating room during the surgery. According to these instructions, the processing device 210 can control the intelligent robotic nurse to deliver the target surgical tools to the surgical participants.

[0387] Image interaction refers to displaying a digital body model of the patient (for example, a three-dimensional anatomical model of the surgical site), an electronic medical record of the patient, a surgical plan for the current surgery, a real-time image of the patient's surgical site, etc. to surgical participants (for example, local surgical participants, remote surgical participants) and / or the patient through an interaction device (for example, a display screen in the operating room, a doctor terminal 270).

[0388] Intraoperative planning and navigation refers to the process of fusing images of the patient's lesion (e.g., CT scan images of the lesion) with the patient's digital body model during surgery, projecting the lesion image onto the patient's body, or overlaying the positioning and tracking of surgical tools to guide surgical participants in the operation.

[0389] Real-time alarms can include behavioral alarms for surgical participants, patient vital sign alarms, and equipment status alarms. Behavioral alarms refer to the monitoring and alerting of surgical participants' intraoperative actions. Patient vital sign alarms can be activated when abnormalities occur in patient vital signs (e.g., electrocardiogram, blood pressure, etc.). Equipment status alarms are alerts issued when surgical equipment malfunctions.

[0390] like Figure 19 As shown, the post-operative process (e.g., post-operative matters) may include steps 1931, 1933, and 1935.

[0391] Step 1931, Patient Transfer. Patient transfer refers to the process of moving the patient from the operating room to the recovery area after the surgical procedure is completed. In some embodiments, the patient transfer may be performed by a healthcare professional assisted by an intelligent robotic nurse.

[0392] Step 1933: Perform operating room cleaning. Operating room cleaning refers to the process of cleaning or disinfecting surgical equipment and instruments. In some embodiments, the processing device 210 can control an intelligent robotic nurse to perform the room cleaning operation.

[0393] Step 1935: Generate the surgical report.

[0394] The surgical report may include surgical-related information, patient-related records, participant-related records, etc. In some embodiments, the processing device 210 may generate an initial surgical report based on data collected during the surgical procedure (e.g., first sensing information collected by one or more first sensing devices in the operating room). The processing device 210 may generate a surgical report based on the initial surgical report and feedback information entered by the physician regarding the initial surgical report.

[0395] In some embodiments, the processing device 210 can also monitor the patient's postoperative vital signs using vital sign monitoring equipment (e.g., electrocardiogram monitor, blood pressure monitor, etc.) in the hospital ward to determine whether the patient's postoperative vital signs are within the normal range, whether there are any abnormalities, or whether the recovery process is normal. Furthermore, the processing device 210 can update the medical recommendation report based on the patient's postoperative vital signs. In some embodiments, the processing device 210 can update the medical recommendation report according to the doctor's instructions. In some embodiments, the processing device 210 can send the updated medical recommendation report to the display device of the nurse's workstation and / or the display device of the doctor's workstation.

[0396] In some embodiments, the processing device 210 can determine a post-operative care plan according to the updated medical recommendation report. The post-operative care plan refers to care tasks that need to be performed by a caregiver (e.g., a nurse, a care assistant, etc.) during the patient’s hospitalization after the surgery. In some embodiments, the processing device 210 can control the smart surgery device (e.g., the smart care cart 240-4) to provide care to the patient according to the post-operative care plan. In some embodiments, the processing device 210 can send the post-operative care plan to a nurse so that the nurse can provide post-operative care to the patient. In some embodiments, the processing device 210 can update the post-operative care plan in real time according to the patient’s condition during the care process. The execution of the post-operative care plan is similar to the daily plan described in Figure 15 .

[0397] In some embodiments, the processing device 210 can generate the surgeon’s surgery outcome and surgery record according to the surgery report and the medical recommendation report, so that the surgeon reviews the surgery process. The surgery outcome refers to data reflecting the result of the surgery process. In some embodiments, the surgery outcome also includes summary data of the surgeon’s surgery outcome in a predetermined time period (e.g., one month). The surgery record refers to the record of the surgeon’s operation during the surgery process. The operation record can include action record, force record, station record, etc. In some embodiments, the processing device 210 can generate the surgery outcome and the surgery record according to the surgery report and the medical recommendation report.

[0398] In some embodiments, the surgery process is reviewed. For example, the processing device 210 can present the surgeon’s surgery outcome and surgery record to the surgeon, allowing the surgeon to review the surgery process.

[0399] Figure 20 is a schematic diagram of an example medical service process 2000 according to some embodiments of the present application. In some embodiments, the processing device 210 (e.g., the service module 430 or an agent corresponding to the patient service configured on the processing device 210) can perform the steps involved in the medical service process 2000.

[0400] At step 2010, patient data is obtained.

[0401] The patient data can be relevant information of a patient receiving medical services. In some embodiments, the data related to the patient can include, but is not limited to, patient location information 2011, patient interaction information 2012, patient examination data 2013, perception data 2014, clinical pathway 2015, and / or data input by a medical service provider 2016.

[0402] The patient location information 2011 is information used to determine the location of the patient at the current time.

[0403] Patient interaction information 2012 can be information generated from the interaction between the patient and the terminal device. For example, personal information, health status, medical history, symptom description, lifestyle habits, and the like input by the patient through the patient terminal and / or the hospital terminal. For another example, patient answers, remote accompanying request, target link selected by the patient, and the like input by the patient through the patient terminal.

[0404] Examination data 2013 is the examination result about the health status of the patient. For example, examination report, medical image, vital sign, and the like of the patient.

[0405] Sensing data 2014 is the information collected by the sensing device. For example, when the patient is in the hospital, the patient data can include the sensing information about the patient collected by one or more sensing devices in the hospital.

[0406] Clinical pathway 2015 refers to the mode and method for specifying standardized medical services, and a corresponding detailed treatment step and management plan can be made according to a specific disease or condition. For example, the clinical pathway can be a time sequence table, which describes in detail the standardized, evidence-based diagnosis and treatment steps for a specific patient group within a specific time period. These steps can include specific operations and goals at each time point corresponding to the processes of diagnosis, treatment, care, and rehabilitation.

[0407] Data input by medical service provider 2016 refers to the patient-related data input by the medical service provider (e.g., doctor, nurse, nursing staff, and the like) through the doctor terminal or the hospital terminal device, such as disease diagnosis, treatment plan, medical order, document modification, and the like.

[0408] In some embodiments, the acquisition module 410 can acquire the data related to the patient through one or more hardware devices, including the patient terminal 2010-a, the doctor terminal 2010-b, the hospital terminal device 2010-c, the examination device 2010-d, and / or the sensing device 2010-e in the environment where the patient is located.

[0409] At step 2020, according to the patient data, it is detected that the patient enters a target link of a medical service process.

[0410] The medical service process is a series of standardized operations and service steps experienced by the patient from the first contact with the medical service to the completion of the treatment and departure. As shown in Figure 20 , the medical service process can include a registration link, a waiting link, an inquiry link, a hospitalization link, a follow-up link, and the like. The target link refers to the medical service link that the patient is about to enter. The detailed description of detecting the patient entering the target link can be found in Figure 21 .

[0411] At step 2030, in response to detecting that the patient enters the target link, the patient terminal is controlled to perform at least one first preset operation corresponding to the target link.

[0412] The first preset operation refers to an operation performed through the interactive interface of the patient terminal. Figure 22 FIG. 1 is a schematic diagram of an example first preset operation according to some embodiments of the present application. As shown in FIG. 1, the first preset operation can include one or more of presenting guidance information 2210, presenting introduction information 2220 of the target link, and / or presenting a service portal 2230. Figure 22

[0413] The guidance information is navigation information of a planned path from the current location of the patient to a target location corresponding to the target link. The planned path can be determined by obtaining an initial 3D map of the hospital and real-time information related to the hospital, generating a real-time 3D map of the hospital according to the initial 3D map and the real-time information, and determining the planned path according to the real-time 3D map of the hospital.

[0414] The introduction information of the target link is information introducing the medical service and process of the target link. In some embodiments, the processing device 210 can present a virtual character explaining the introduction information. Alternatively, the processing device 210 can receive a question about the target link from the patient terminal after the introduction information, and determine the answer to the question. The processing device 210 can cause the patient terminal to present the answer to the patient.

[0415] The service portal is an entrance for the user to obtain medical services at the target link and / or a link after the target link. In some embodiments, the processing device 210 can cause the patient terminal to present guidance information related to a planned path from the current location of the patient to a target location corresponding to the target link, and determine whether the patient reaches the target location according to the patient-related data. In order to determine whether the patient reaches the target location, the processing device 210 can cause the patient terminal to present a service portal for accessing user services related to the target link.

[0416] In some embodiments, the patient terminal includes an XR device, and the processing device 210 can cause the XR device to overlay information related to the first preset operation on the patient's real-world view through AR technology or MR technology. In some embodiments, the patient terminal has a patient space application. The patient interaction information 2012 between the patient and the patient terminal can be obtained using the patient space application. The first preset operation can be performed using the patient space application.

[0417] In some embodiments of the present application, when it is detected that the patient enters the target link, the patient terminal is controlled to perform the first preset operation according to the target link, to provide the patient with medical services corresponding to the target link, and to help the patient participate in the required medical service process, thereby improving the efficiency and quality of medical services.​

[0418] In some embodiments, in response to detecting that the patient switches from the current medical service session to the target session, the control physician terminal device performs at least one second preset operation.

[0419] The second preset operation is an operation performed by an interactive interface of the physician terminal device. In some embodiments, a doctor space application is installed in the physician terminal device. The second preset operation can be performed by using the doctor space application.

[0420] In some embodiments, the second preset operation can include presenting prompt information indicating that the patient has completed the current session, presenting patient records related to the patient's current session, and / or presenting patient schedules or patient requests related to the patient's service in the target session.

[0421] In some embodiments, in response to detecting that the patient's session switches from the current medical service session to the target session, the control hospital terminal device performs at least one third preset operation.

[0422] The third preset operation is an operation performed by an interactive interface of the hospital terminal device. In some embodiments, a tube space application is installed on the hospital terminal device. The third preset operation can be performed by using the tube space application. In some embodiments, the processing device 210 can cause the public terminal device in the hospital to perform at least one third preset operation.

[0423] In some embodiments, the third preset operation can include presenting patient records related to the patient's current session, presenting prompt information indicating that the patient has completed the current session, presenting patient schedules or patient requests related to the patient's service in the target session, and / or presenting service reservation information of the patient's target session.

[0424] In some embodiments, in response to detecting that the patient's session switches from the current medical service session to the target session, the processing device 210 can schedule service resources for the medical service of the patient's target session. For example, when detecting that the patient enters the examination session, the processing device 210 can allocate an examination room and an examination technician for the patient.

[0425] In some embodiments, if the target session is an outpatient registration session, the at least one preset operation can include conducting a first inquiry on the patient to determine the doctor to which the patient registers.

[0426] In some embodiments, if the target session is an outpatient waiting session, the at least one preset operation can include conducting a second inquiry on the patient to make a pre-consultation.

[0427] In some embodiments, if the target session is a follow-up session, the at least one preset operation can include reminding the patient of the follow-up at a scheduled time.

[0428] Figure 21 FIG. 21 is a schematic diagram of an exemplary process 2100 for determining a target link according to some embodiments of the present disclosure.

[0429] At step 2110, the processing device 210 detects that a current link of the medical service process has been completed according to the patient data.

[0430] The current link is a medical service link that the patient is currently involved in. For example, the processing device 210 can detect that the current link has been completed based on the patient location information 2011. For example, when the medical service system 200 provides on-site medical services, the processing device 210 determines that the patient has left the location corresponding to the current link according to the patient location information 2011, and then determines that the current link has been completed.

[0431] As another example, the processing device 210 can detect that the current link has been completed based on the patient interaction information 2012. As another example, the processing device 210 can detect that the current link has been completed according to the medical service provider input data 2016. As another example, the processing device 210 can detect that the current link has been completed according to the examination data 2013. As another example, the processing device 210 can detect that the current link has been completed according to the clinical pathway 2015. As another example, the processing device 210 can detect that the current link has been completed based on the perception data 2014.

[0432] At step 2020, one or more next links that the patient is likely to perform are predicted.

[0433] In some embodiments, the processing device 210 can determine one or more next links that the patient is likely to perform according to a standardized operation process of the medical service process. The standardized operation process is a medical service process applicable to most patients. For example, after detecting that the patient has completed the registration link, the processing device 210 can determine that the next link that the patient is likely to perform is the waiting link based on the standardized operation process of the outpatient service.

[0434] In some embodiments, the processing device 210 can determine one or more next links that the patient is likely to perform according to a patient individualized process of the medical service process. The patient individualized process refers to a medical service process for special patients (e.g., VIPs, critically ill patients). In some embodiments, the processing device 210 can determine one or more next links that the patient is likely to perform according to the patient's out-of-hospital health information.

[0435] In some embodiments, when there are multiple next steps that the patient can perform, the processing device 210 can recommend the next step according to the time for the patient to go to the location corresponding to the next step and the estimated waiting time of the patient. For example, for each of the multiple next steps, the processing device 210 can determine the first estimated time from the current location to the target location of the corresponding next step and the second estimated time for waiting at the target location according to the real-time 3D map of the hospital. In addition, the processing device 210 can determine at least part of the multiple next steps based on the first and second estimated times of each next step.

[0436] Step 2130, control the patient terminal to present at least part of one or more next steps that the patient can perform.

[0437] The patient terminal 2010-a can present one or more next steps that the patient can perform, or the next step that the patient is recommended to enter preferentially by the processing device 210. For example, after completing the registration step, the patient terminal 2010-a can present multiple next steps that can be performed, such as the waiting step, the consultation step, and the hospitalization step.

[0438] Step 2140, obtain the target step selected by the patient from the patient terminal.

[0439] The patient can select the target step by text input, voice input, screen click, gesture input, etc. through the patient terminal. In some embodiments of the present application, after detecting that the current step is completed, the next step that can be performed can be automatically presented to the patient according to different medical processes corresponding to different patients, helping the patient to understand the medical process, and at the same time giving the patient the freedom to choose. In addition, personalized services are also provided for the patient.

[0440] Figure 23 is a schematic flowchart of an exemplary process 2300 for assisting doctors according to some embodiments of the present application. In some embodiments, the process 2300 can be implemented by the processing device 210 (for example, the service module 430 or an intelligent agent corresponding to the doctor service configured on the processing device 210).

[0441] Step 2310, an access request for a doctor space application can be obtained from the doctor terminal.

[0442] The access request refers to a request initiated by the physician to access the medical space application. The physician can generate the access request in various ways. For example, the medical space application can be displayed on the display screen of the mobile terminal device 270-1, and the physician 271 can generate the access request by clicking on the icon of the medical space application on the display screen or by issuing a voice command (e.g., “start the medical space application”). As another example, the XR device 270-2 can generate a virtual reality version of the medical space application, and the physician 271 can generate the access request by interacting with the virtual reality medical space application or using a voice command.

[0443] At step 2320, in response to the access request, one or more pending tasks of the physician can be determined according to the receiving time of the access request and the schedule information of the physician.

[0444] The receiving time of the access request refers to the time point when the processing device 210 receives the access request sent from the physician terminal 270.

[0445] The schedule information of the physician refers to the details of the work arrangement of the physician for the day. The schedule information of the physician can include various work tasks that the physician needs to complete for the day, and the time corresponding to each work task (e.g., the planned start time and the planned end time of each work task). By way of example only, the schedule information of the physician can be “ward round preparation (7:45-8:00), ward round (8:00-9:00), pre-consultation preview (9:00-9:10), consultation (9:10-11:30), preoperative preparation (13:30-14:00), surgery (14:00-17:00)”.

[0446] The pending task refers to a task that has not been completed by the physician for the day, which can include that the physician is currently processing a task that has not started, or a combination thereof. For example, taking the schedule information of the physician in the above example, if the physician is currently conducting the consultation, the one or more pending tasks can include the consultation, the preoperative preparation, and the surgery. In some embodiments, the processing device 210 can determine one or more tasks whose planned end times are later than the receiving time of the access request as the pending tasks.

[0447] In some embodiments, the one or more pending tasks can include conducting a ward round in a hospital ward. In some embodiments, the physician can conduct the ward round by participating in the ward round remotely. For example, the processing device 210 can obtain a request to participate in the ward round remotely input by the physician from the physician terminal. In response to detecting that the ward round is conducted in the hospital ward, the processing device 210 can obtain perception information collected by one or more perception devices in the hospital ward during the ward round, generate a virtual ward space based on the perception information and the patient data, and cause the physician terminal to present the virtual ward space.

[0448] In some embodiments, the one or more pending tasks can include providing a service of a medical consultation in a clinic. In some embodiments, the one or more pending tasks can include providing a service of a remote medical consultation. The service of a remote medical consultation refers to a medical diagnosis and consultation service provided by a doctor through an online platform (e.g., the DoctorSpace application). In some embodiments, the one or more pending tasks can include performing a surgery on a target patient.

[0449] In some embodiments, the one or more pending tasks can include writing a work record. The work record can record details of a doctor’s daily activities, such as the work content completed on the day, the working hours, the emergency during the task, the task summary, etc. In some embodiments, the work record can include a daily work record, a work record for each task, or a work record for a preset time period (e.g., 3 hours, 5 hours, 24 hours, etc.). In some embodiments, the one or more pending tasks of writing a work record can be displayed at all times.

[0450] In some embodiments, the one or more pending tasks can include reviewing records of one or more completed tasks. For example, the doctor 271 can access records related to a ward round, a medical consultation, a surgery, etc. through the doctor terminal 270 and view the records. When viewing the records of the one or more completed tasks, the doctor can add, modify, and / or delete content in the records.

[0451] In some embodiments, the processing device 210 can determine one or more completed tasks of the doctor according to the receiving time and the schedule information of the doctor. In some embodiments, the processing device 210 can determine a task as a completed task if the end time of the task is earlier than the receiving time of the access request. In some embodiments, since some tasks can be completed in advance, the doctor can input a task completion instruction (e.g., a voice command “medical consultation task completed”) through the doctor terminal 270 according to the actual completion status of the task. The doctor terminal 270 can send the task completion instruction to the processing device 210, and the processing device 210 can determine the corresponding task as a completed task according to the task completion instruction.

[0452] In some embodiments, in response to the access request, the processing device 210 can cause the doctor terminal 270 to present an initial interaction interface including an eighth interface element for reminding the doctor to check the work schedule (i.e., the schedule information of the doctor) through the DoctorSpace application. In some embodiments, in response to a request of the doctor to access the work schedule input through the doctor terminal, the processing device 210 can determine the one or more pending tasks.

[0453] At step 2330, the doctor terminal can be caused to present an interaction interface through the DoctorSpace application. For example, as shown in FIG. 23B, the interaction interface 2300 can be presented on the doctor terminal 270. The interaction interface 2300 can include a first interface element 2301 for checking the work schedule of the doctor, a second interface element 2302 for checking the work record of the doctor, a third interface element 2303 for checking the records of the one or more completed tasks, and a fourth interface element 2304 for checking the records of the one or more pending tasks. Figure 23As shown, the processing device 210 can cause the physician terminal 270 to present an interaction interface 2331. The interaction interface 2331 can be presented in a manner that: if the physician terminal 270 is a terminal device with a display screen (e.g., the mobile terminal device 270-1 or the desktop terminal device, etc.), the interaction interface 2331 can be directly presented on the display screen; if the physician terminal 270 is the XR device 270-2, the XR device 270-2 can present the interaction interface 431 in a virtual reality space generated by the XR device 270-2. More description about the interaction interface can be found in Figure 24 and related descriptions thereof.

[0454] The interaction interface can include at least one interface element. The physician can access one or more assistance services corresponding to the at least one interface element by accessing the at least one interface element. The physician can access the at least one interface element by clicking, long-pressing, voice selecting, etc.

[0455] The assistance service refers to a function provided by the medical space application to assist the physician in completing a work task. For example, the assistance service can include displaying patient data, displaying a 3D map of a target location in the hospital, and providing a service for the physician to remotely participate in a medical examination or a ward round.

[0456] In some embodiments, the interaction interface can include a first interface element for accessing an assistance service related to at least one of the one or more pending tasks. The physician can access the assistance service corresponding to the first interface element by clicking or selecting the first interface element in the voice. More description about the interaction interface can be found in Figure 24 and related descriptions thereof.

[0457] Figure 24 is a schematic diagram of an exemplary interaction interface 2331 according to some embodiments of the present application.

[0458] In some embodiments, as Figure 24 shown, the interaction interface 2331 can include a first interface element 2410 (hereinafter referred to as a plurality of first interface elements 2410) for accessing an assistance service related to at least one of the one or more pending tasks.

[0459] In some embodiments, when the one or more pending tasks include a ward round in a hospital ward, the first interface element can include a first interface element for applying for a remote participation in the ward round (hereinafter referred to as a ward round interface element). For example, as Figure 24 shown, the plurality of first interface elements 2410 can include a ward interface element 2411.

[0460] In some embodiments, when the one or more pending tasks include providing an outpatient service in a clinic room, the first interface elements can include a first interface element for accessing patient data of a patient who has scheduled an appointment for the outpatient service (hereinafter referred to as an appointment interface element). For example, as shown in Figure 24 FIG. 24A, the plurality of first interface elements 2410 can include an appointment interface element 2412. In some embodiments, the first interface elements further include a first interface element for accessing an initial diagnosis record related to the outpatient service, which is generated based on perceptual information collected by one or more perceptual devices in the clinic room during the outpatient service. For example, the processing device 210 can obtain, from the doctor terminal, a request to access patient data of a target patient among the patients, generate a virtual character representing the target patient according to the patient data of the target patient, and cause the doctor terminal to present the virtual character to explain the patient data of the target patient to the doctor.

[0461] In some embodiments, when the one or more pending tasks include providing a remote outpatient service, the first interface elements can include a first interface element for entering a virtual clinic room (hereinafter referred to as a clinic room interface element). For example, as shown in Figure 24 FIG. 24B, the plurality of first interface elements 2410 can include a clinic room interface element 2413. In response to an interaction between a doctor (e.g., the doctor 271) and the clinic room interface element 2413, the processing device 210 can cause the doctor terminal 270 (e.g., the XR device 270-2) to present a virtual clinic room. For example, the processing device 210 can obtain, from the doctor terminal, a request to enter the virtual clinic room to provide a remote outpatient service to a target patient, and cause the doctor terminal to present a 3D patient model of the target patient. The processing device 210 can obtain, from the doctor terminal, examination instructions input by the doctor through interaction with the 3D patient model, and cause a wearable device worn by the target patient to collect examination data of the target patient according to the examination instructions.

[0462] In some embodiments, when the one or more pending tasks include performing surgery on a target patient, the first interface elements can include a first interface element for accessing patient data related to the target patient (hereinafter referred to as a surgery interface element). For example, as shown in Figure 24 FIG. 24C, the plurality of first interface elements 2410 can include a surgery interface element 2414 for accessing patient data related to the target patient. The patient data can include data related to a target surgery plan corresponding to the target patient.

[0463] In some embodiments, the first interface elements can further include a first interface element for updating a medical order of the target patient. In some embodiments, the first interface elements can further include a first interface element for accessing an initial surgery record of the surgery. The initial surgery record can be generated based on perceptual information collected by one or more perceptual devices in the operating room during the surgery.

[0464] In some embodiments, as shown in FIG. 23A, the interactive interface 2331 can further include a second interface element 2320 for accessing a real-time 3D map of the target location corresponding to the at least one pending task (i.e., to browse the target location corresponding to the at least one pending task). The real-time 3D map refers to a virtual reality (VR) model of the space corresponding to the target location. In some embodiments, the real-time 3D map of the target location can be generated based on the initial 3D map of the hospital and real-time information of the target location. Figure 24 In some embodiments, as shown in FIG. 23A, the interactive interface 2331 can further include a third interface element 2430 for conducting preoperative education. In response to an interaction between the doctor and the third interface element 2430, the doctor terminal 270 can generate a request for preoperative education of the target patient and send the request to the processing device 210. For example, the processing device 210 can obtain the request for preoperative education of the target patient. The request can be obtained from the XR device of the doctor and input by the doctor through interaction with the third interface element. In response to the request, the processing device 210 can generate a lecture material for explaining the candidate surgical plan of the target patient and cause the XR device worn by the patient and the XR device of the doctor to present the lecture material to the target patient and the doctor simultaneously.

[0465] Figure 24 In some embodiments, as shown in FIG. 23A, the interactive interface 2331 can further include a fourth interface element 2440 for conducting surgical simulation. In response to an interaction between the doctor and the fourth interface element 2440, the doctor terminal 270 can generate a request for simulating the target surgery and send the request for simulating the target surgery to the processing device 210. The target surgery refers to a surgery corresponding to the target surgical plan. For example, the processing device 210 can obtain the request for simulating the target surgery. The request can be obtained from the XR device of the doctor and input by the doctor through interaction with the fourth interface element. In response to the request, the processing device 210 can generate a virtual surgery scene corresponding to the target surgery. The virtual surgery scene can include a virtual surgical site and virtual surgical devices. The processing device 210 can cause the XR device of the doctor to present the virtual surgery scene to the doctor. In some embodiments, the processing device 210 can obtain interaction instructions input by the doctor on the virtual surgical devices through the XR device of the doctor or an interactive device corresponding to the virtual surgical devices, and update the virtual surgical site and the virtual surgical devices in the virtual surgery scene based on the interaction instructions.

[0466] In some embodiments, as shown in FIG. 23A, the interactive interface 2331 can further include a fourth interface element 2440 for conducting surgical simulation. In response to an interaction between the doctor and the fourth interface element 2440, the doctor terminal 270 can generate a request for simulating the target surgery and send the request for simulating the target surgery to the processing device 210. The target surgery refers to a surgery corresponding to the target surgical plan. For example, the processing device 210 can obtain the request for simulating the target surgery. The request can be obtained from the XR device of the doctor and input by the doctor through interaction with the fourth interface element. In response to the request, the processing device 210 can generate a virtual surgery scene corresponding to the target surgery. The virtual surgery scene can include a virtual surgical site and virtual surgical devices. The processing device 210 can cause the XR device of the doctor to present the virtual surgery scene to the doctor. In some embodiments, the processing device 210 can obtain interaction instructions input by the doctor on the virtual surgical devices through the XR device of the doctor or an interactive device corresponding to the virtual surgical devices, and update the virtual surgical site and the virtual surgical devices in the virtual surgery scene based on the interaction instructions. Figure 24 In some embodiments, as shown in FIG. 23A, the interactive interface 2331 can further include a fourth interface element 2440 for conducting surgical simulation. In response to an interaction between the doctor and the fourth interface element 2440, the doctor terminal 270 can generate a request for simulating the target surgery and send the request for simulating the target surgery to the processing device 210. The target surgery refers to a surgery corresponding to the target surgical plan. For example, the processing device 210 can obtain the request for simulating the target surgery. The request can be obtained from the XR device of the doctor and input by the doctor through interaction with the fourth interface element. In response to the request, the processing device 210 can generate a virtual surgery scene corresponding to the target surgery. The virtual surgery scene can include a virtual surgical site and virtual surgical devices. The processing device 210 can cause the XR device of the doctor to present the virtual surgery scene to the doctor. In some embodiments, the processing device 210 can obtain interaction instructions input by the doctor on the virtual surgical devices through the XR device of the doctor or an interactive device corresponding to the virtual surgical devices, and update the virtual surgical site and the virtual surgical devices in the virtual surgery scene based on the interaction instructions.

[0467] Figure 24 ​​As shown, the interactive interface 2331 may include a fifth interface element 2450 for executing a surgical plan. In response to the interaction between the physician and the fifth interface element 2450, the physician terminal 270 may generate a request to perform a surgical plan for the target patient and send the request to the processing device 210. For example, the processing device 210 may receive a request to perform a surgical plan for the target patient. This request may be obtained from the physician terminal and entered by the physician through interaction with the fifth interface element. The processing device 210 may determine the surgical difficulty coefficient based on the target patient's patient data and determine whether an expert meeting is required based on the surgical difficulty coefficient. In response to determining that an expert meeting is required, the processing device 210 may cause the physician terminal to display a sixth interface element 2460 for initiating an expert meeting.

[0468] In some embodiments, such as Figure 24 As shown, the interactive interface 2331 may further include a seventh interface element 2470 for patient management. Patient management refers to the management of patient-related data (e.g., patient data, surgical records, hospitalization records, nursing records, postoperative recovery records, etc.). In some embodiments, in response to an interaction between a doctor and the seventh interface element 2470, the doctor's terminal 270 may generate a request to access the initial hospitalization record of a target patient and send the request to the processing device 210. For example, the processing device 210 may receive a request to access the initial hospitalization record of a target patient. The request can be obtained from the doctor's terminal and entered by the doctor through interaction with the seventh interface element. In response to the request, the processing device 210 may cause the doctor's terminal to present the initial admission record and update the initial admission record based on feedback information about the initial admission record entered by the doctor through the doctor's terminal.

[0469] In some embodiments, in response to an access request, processing device 210 may cause a doctor's terminal to present an initial interactive interface via a medical space application, which includes an eighth interface element for reminding the doctor to check their work schedule. In response to a doctor's request to access their work schedule via their terminal, processing device 210 may identify one or more pending tasks. In some embodiments, the initial interactive interface may further present a virtual avatar configured to communicate with the doctor.

[0470] In some embodiments, the interface 2331 may further include schedule information related to at least one task to be processed. For example, such as Figure 24 As shown, the interactive interface 2431 may include schedule information 2480 such as ward rounds (scheduled start time can be 8:00), pre-consultation / pre-consultation (scheduled start time can be 9:00), and consultation (scheduled start time can be 9:10).

[0471] In some embodiments, the interactive interface 2331 can further include one or more foldable elements related to one or more completed tasks. The one or more foldable elements in the interactive interface can be folded by human-computer interaction (e.g., clicking or selecting in voice, etc.), zoomed out (e.g., zoomed out to 1 / 4 of the original size), or zoomed in. Thus, the foldable element can include a folded state (a state when zoomed out) and an unfolded state (a state when zoomed in). In the unfolded state, the foldable element can display the completed task and time information (e.g., the start time of the one or more completed tasks) of the completed task. For example, as shown in Figure 24 FIG. 23B, the interactive interface 2331 can include a foldable element 2490 (in the unfolded state), which can present a ward round preparation task, and the start time of the ward round preparation task can be 7:45. The doctor can click the foldable element 2490 on the interactive interface 2331, and the processing device 210 can make the foldable element 2490 enter the folded state (zoom out the area of the foldable element 2490, and no longer present the completed task) according to the clicking operation.

[0472] In some embodiments, the configuration of the interactive interface 2331 can be determined according to the preference information of the doctor. The preference information reflects the doctor’s preference for the content, specifications (including shape, color, font, etc.), and / or location of the elements (e.g., first interface element, second interface element, etc.) on the interface. In some embodiments, the preference information can also reflect the doctor’s preference for the interface style. The interface style can include a simple style and a detailed style. The simple style can contain fewer interface elements than the detailed style. In some embodiments, the preference information can include whether to display a virtual character.

[0473] Figure 24 FIG. 25 is a schematic diagram of an exemplary system 2500 (also referred to as a hospital management system 2500) for hospital management according to some embodiments of the present application. As shown in Figure 25 FIG. 25, the hospital management system 2500 can include managers 2510, a space management application 2520, and resources 2530 of a hospital.

[0474] The managers 2510 can manage the hospital resources 2530 through the space management application 2520. The managers 2510 can include managers of the hospital, such as the president of the hospital, the supervisor of the department of the hospital (e.g., the department of stomatology, the department of internal medicine, etc.), and the like. The hospital resources 2530 can include devices, personnel, digital twins, digital intelligent resources (e.g., intelligent agents), and the like, or any combination thereof. In some embodiments, different managers 2510 can have different management permissions for different resources 2530. In some embodiments, a particular manager 2510 only manages a particular resource 2530. The space management application 2520 can be presented through the display of the terminal device of the manager 2510.

[0475] In some embodiments, the management space application 2520 may present an interface. The interface can be presented via a display of the administrator terminal device of the administrator 2510. The display may include a light-emitting diode (LED) display, a liquid crystal display, an electronic ink display, a touch LCD, an organic LED touch, or any combination thereof. The terminal device may include a mobile device, an XR device, a smart wearable device, etc.

[0476] In some embodiments, the management space application 2520 (or its interface) can be configured to present a virtual avatar to assist the administrator 2510 in managing resources. The virtual avatar can be configured to communicate with the administrator 2510. The management space application 2520 can also be configured to provide interface elements to the administrator 2510 to initiate a communication session with the virtual avatar. In some embodiments, the administrator 2510 can initiate a communication session with the virtual avatar through interface elements to express their resource management needs (e.g., viewing information related to a specific type of resource, scheduling a specific type of resource, updating relevant parameters of a specific type of resource) via voice, text, gestures, etc. A processing device (e.g., processing device 210) can analyze user input (e.g., using AI technology, using an intelligent agent), determine feedback information, and convey the feedback information to the administrator 2510 through the virtual avatar during the communication session. In some embodiments, the content displayed by the management space application 2520 can be updated based on the communication content of the communication session.

[0477] Traditional hospital management systems typically have interfaces limited to providing users with pre-defined analytical results. The hospital management system described in this application, however, allows users to retrieve data across different dimensions. Furthermore, virtual avatars allow users to express their needs in natural language, thereby enhancing the user experience and improving the quality and efficiency of resource management services.

[0478] In some embodiments, such as Figure 25 As shown, hospital resource 2530 includes a digital twin 2531. A digital twin can map the state of a corresponding physical entity and is generated according to a predefined data structure. For example, as used herein, a digital twin refers to a virtual representation (or digital copy) of a physical entity. A physical entity refers to any object or phenomenon existing in the physical world that can be directly or indirectly observed, measured, or interacted with.

[0479] In some embodiments, the preset data structure of the digital twin can specify a format of the digital twin, a type of information reflected by the digital twin, a storage address of the digital twin, an access portal of the digital twin, an update mode of the digital twin, a modification permission of the digital twin, a modification permission of the preset data structure, or any combination thereof. In some embodiments, the preset data structure can be a default data structure provided by the hospital, or a customized data structure set by the administrator 2510 through the tube space application 2520.

[0480] In some embodiments, the digital twins 2531 can include one or more first digital twins 25311. For each first digital twin, the mapping of the state of the corresponding physical entity can include updating the first digital twin based on an update of the state of the corresponding physical entity. Exemplary first digital twins can include a digital twin corresponding to a public area of the hospital, a digital twin corresponding to a medical service, a digital twin corresponding to a user (e.g., a patient, an organ of a patient, a doctor), a digital twin corresponding to a hardware device of the hospital, and the like. In some embodiments, the update of the state of the corresponding physical entity is detected according to real-time information of the corresponding physical entity. For example, the real-time information of the corresponding physical entity includes at least one of information collected by a sensing device within the hospital or information collected by a user terminal associated with the hospital.

[0481] In some embodiments, the digital twins 2531 can include one or more second digital twins 25312. For each second digital twin, the mapping of the state of the corresponding physical entity can include updating the corresponding physical entity based on an update of the second digital twin. Exemplary second digital twins can include a digital twin corresponding to a hardware device, a digital twin corresponding to a user service, a digital twin corresponding to a medical service process, and the like. It should be understood that a digital twin can be a first digital twin or a second digital twin.

[0482] In some embodiments, the administrator 2510 can manage the digital twins 2531 through the tube space application 2520. Since the digital twins map the states of the corresponding physical entities, the physical entities of the hospital can be managed by managing the digital twins 2531. For example, the administrator 2510 can view the first digital twins 25311 of the physical entities (e.g., 3D digital twin models of patient organs or hardware devices, digital twin views of public areas) through the tube space application 2520 to understand and evaluate the states of the physical entities. Optionally, the user can change the display angle, display size, and the like of the displayed first digital twins 25311.

[0483] In some embodiments, the one or more first digital twins 25311 can include digital twins corresponding to public areas in the hospital. Based on the digital twins corresponding to the public areas of the hospital, monitoring of the public areas can be implemented. In some embodiments, the digital twins corresponding to the public areas can reflect digital twin views of the public areas generated based on real-time information of the public areas. The digital twin views can be used to monitor, analyze, and predict the performance and operation of the public areas in real time, so that insights into the current status of the public areas can be provided. In some embodiments, the digital twin views of the public areas can present real-time 3D maps of the public areas and monitoring indicators of the public areas. The monitoring indicators can include monitoring indicators related to users within the public areas, monitoring indicators related to events within the public areas, monitoring indicators related to devices within the public areas, and the like, or any combination thereof.

[0484] In some embodiments, the one or more first digital twins 25311 can include digital twins corresponding to medical services. The digital twins corresponding to the medical services can be used to perform medical service evaluations of the medical services. For example, the digital twins corresponding to the medical services can reflect operational indicators of the medical services, which are used to evaluate the quality, efficiency, profit, and the like of the medical services.

[0485] In some embodiments, the one or more first digital twins 25311 can include other digital twins that are updated based on status updates of the corresponding physical entities. By way of example only, the first digital twins can include digital twins of patients or patient organs, which are updated as soon as new medical data (e.g., new medical images, new examination data) of the patients are collected. By way of another example, the first digital twins can include digital twins of hardware devices, which are updated as soon as usage status, operational parameters, and the like of the hardware devices are updated.

[0486] In some embodiments, for each of at least a portion of the one or more first digital twins 25311, the first digital twin can be updated in a first manner in response to detecting that a status update of the corresponding physical entity is normal, and the first digital twin can be updated in a second manner in response to detecting that the status update of the corresponding physical entity is abnormal. The first manner can be different from the second manner. In other words, the first digital twin can be updated in different manners when abnormal and normal status changes of the corresponding physical entity occur. For example, in response to detecting that the status update of the corresponding physical entity is normal, the first digital twin can display a first marker symbol (or a first color). In this way, the first digital twin can be used to monitor the physical entity for abnormalities, facilitating timely adjustments to the physical entity.

[0487] In some embodiments, the one or more second digital twins 25312 can include a digital twin corresponding to a hardware device, and the digital twin corresponding to the hardware device reflects parameters of the hardware device. The second digital twin corresponding to the hardware device can be used to update / set the parameters of the hardware device.

[0488] In some embodiments, the hardware device can include a display device that displays information related to a medical service. The digital twin corresponding to the hardware device can reflect display parameters of the display device, which can be set or updated by the administrator 2510 through the pipe space application 2520. For example, by updating the digital twin corresponding to the hardware device, the administrator 2510 can adjust the display content of the display device.

[0489] In some embodiments, the one or more second digital twins 25312 can include a digital twin corresponding to a user service, and the digital twin corresponding to the user service reflects parameters of the user service. In some embodiments, the parameters of the user service can be updated / set using the digital twin corresponding to the user service. Exemplary parameters of the user service can include a way of providing the user service, a requirement of a user using the user service, content of the user service, and the like, or any combination thereof. In some embodiments, the user service can be accessed from a patient terminal installed with a patient space application or a doctor terminal installed with a doctor space application.

[0490] In some embodiments, the one or more second digital twins 25312 can include a digital twin corresponding to a medical service process, and the digital twin corresponding to the medical service process reflects parameters of the medical service process. In some embodiments, the parameters of the medical service process can include a standard operating procedure (SOP) that specifies standard steps in the medical service process. In some embodiments, the SOP can further specify a preset data acquisition protocol. In some embodiments, the second digital twin corresponding to the medical service process can be configured to update / set the parameters of the medical service process.

[0491] Digital twins can be instantiated with data obtained from sensors and other sources to model corresponding real-world entities in real time dynamically. In accordance with the hospital management system disclosed herein, digital twins can be deployed for monitoring, analysis, simulation, and control to provide valuable insights to optimize performance and improve overall efficiency of the hospital management system.

[0492] In some embodiments, the hospital resources 2530 include digital intelligent resources of the hospital. Digital intelligent resources can include various types of digital assets and tools enhanced by artificial intelligence to support and improve different aspects of digital operations, learning, and management. In some embodiments, as Figure 25 Figure 25As shown, the digital smart resources can include smart agents 2532. In some embodiments, the manager 2510 can manage the smart agents 2532 through the tube space application 2520. For example, the manager 2510 can view and modify information related to the smart agents through the tube space application 2520. Since the smart agents 2532 are involved in processing data to implement user services, the user services can be managed by managing the smart agents 2532.

[0493] In some embodiments, the tube space application 2520 can present at least a portion of the base configuration data used by the smart agents 2532. The base configuration data can be updated by the manager 2510 through the tube space application 2520. The base configuration data includes key information relied on by the smart agents in providing certain services. Exemplary base configuration data can include at least one of a dictionary (e.g., a list of words, a directory of personnel), a knowledge database, or a template.

[0494] In some embodiments, the tube space application 2520 can present at least a portion of the running metrics of the smart agents 2532. The running metrics can reflect information related to the quantity and quality of the services provided by the smart agents. The running metrics of the smart agents can include the number of users served by the smart agents, the number of services provided by the smart agents, the amount of data processed by the smart agents, the quality of service of the smart agents, etc., or any combination thereof.

[0495] In some embodiments, the smart agents 2532 include smart agents corresponding to different types of medical service providers, different hospital departments, different medical service processes, different user services, etc. According to some embodiments of the present application, the configurations of various smart agents in the hospital can be customized, the adaptability and application range of the smart agents can be enhanced, and thus the accuracy and efficiency of the user services supported by the smart agents can be significantly improved, while the treatment experience of the patients is enhanced.

[0496] In some embodiments, the hospital management system 2500 can further include a processing device. The processing device can process data and / or information obtained from the hospital management system 2500. In some embodiments, the processing device can receive information and / or instructions input by the manager 2510 through the tube space application 2520, and provide corresponding feedback after processing the information and instructions. For example, the processing device 210 can receive update information related to a digital twin corresponding to a hardware device from a manager terminal presenting the tube space application 2520, and control the hardware device to update its configuration based on the update information. For example, the processing device 210 can store the update information to a storage device, and send an update notification to the hardware device, so that the hardware device obtains the update information from the storage device to update its configuration. As another example, the processing device 210 can receive update information related to a smart agent through the tube space application 2520, and control the smart agent to perform a certain operation.

[0497] The basic concepts have been described above. It is clear that the detailed disclosure above is merely illustrative and does not constitute a limitation of this specification. Although not explicitly stated herein, various modifications, improvements, and corrections may be made to this specification by those skilled in the art. Such modifications, improvements, and corrections are suggested in this specification and therefore remain within the spirit and scope of the exemplary embodiments described herein.

[0498] Furthermore, this specification uses specific terms to describe embodiments thereof. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Moreover, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0499] Furthermore, unless expressly stated in the claims, the order of elements and sequences, the use of numbers and letters, or other names in this specification are not intended to limit the order of the processes and methods described herein. Although various examples have been discussed in the foregoing disclosure of some embodiments that are currently considered useful, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, they are intended to cover all modifications and equivalent combinations that conform to the spirit and scope of the embodiments described herein. For example, while the system components described above can be implemented using hardware devices, they can also be implemented solely using software solutions, such as installing the described system on an existing server or mobile device.

[0500] Similarly, it should be noted that, in order to simplify the description disclosed herein and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of embodiments in this specification may sometimes combine multiple features into a single embodiment, drawing, or description thereof. However, this method of disclosure does not imply that the subject matter of this specification requires more features than those mentioned in the claims. In fact, the embodiments contain fewer features than all the features of a single embodiment disclosed above.

[0501] In some embodiments, numbers that describe amounts, dimensions, and so forth, are used in the description of the embodiments. Such numbers are to be understood as being modified in all instances by a term or terms depicting the approximate nature of the value. For example, "substantially" can mean plus or minus 20% of the value. Thus, in some embodiments, numerical parameters in the specification and claims are approximations. Although the numerical parameters are approximations, the numerical and functional aspects of the embodiments described herein are deemed as being approximations. Numerical parameters are thus merely intended to convey the approximate, significant digit data supplied by a given number. Any numerical values, however, are not to be construed in a limiting sense as modifications are expected to occur to those skilled in the art.

[0502] Each patent, patent application, publication, and other material cited in this specification is hereby incorporated by reference in its entirety for the teachings relevant to the sentence and / or paragraph in which the reference is presented. Document(s) incorporated by reference into this patent file are to be taken as being presented in its entirety into this patent file and can only be disposed of by amending the patent file. To the extent such description, definitions, and / or terminology in the cited materials conflict with that of the present description, those in the present description prevail. In the event that any conflicts with the definitions and the terminology used herein do arise, the definition and terminology used herein are intended to govern.

[0503] Finally, it should be understood that the embodiments described herein are merely exemplary of the principles of the present description. Other variations having essentially the same structure and function are within the scope of the present description. Accordingly, the embodiments described herein are not to be taken in a limiting sense, but are merely for illustrating illustrative aspects of the present description. Accordingly, it is not intended that the scope of the present description be limited to the examples described herein.

Claims

1. A medical service system, comprising: Hardware devices are configured to collect data related to the physical entities of a physical hospital, the physical entities including at least the hardware devices and medical service processes of the physical hospital. The processing equipment is configured as follows: The data is mapped to a virtual hospital corresponding to the physical hospital, the virtual hospital including a digital twin corresponding to the physical entity of the physical hospital, at least a portion of the digital twin being updated during the data mapping process; as well as User services are provided to relevant users of the physical hospital through user space applications, wherein... The user space application is configured to provide relevant users of the physical hospital with an access point to interact with the virtual hospital. At least a portion of the user services are provided to the relevant user based on the interaction between the relevant user and at least one of the digital twins.

2. The medical service system according to claim 1, wherein, The physical entity also includes the relevant users and / or environment of the physical hospital.

3. The medical service system according to claim 1, wherein, The data includes real-time information relating to the physical entities corresponding to one or more first digital twins in the digital twins, and the updating of one or more first digital twins in the data mapping process includes: For each first digital twin Based on real-time information related to the physical entity corresponding to the first digital twin, the state update of the physical entity is detected, wherein the real-time information includes information collected by sensing devices in the physical hospital and / or information collected by user terminals associated with the physical hospital; and The state of the first digital twin is updated based on the state update of the physical entity.

4. The medical service system according to claim 1, wherein, When the relevant user interacts with the virtual hospital, at least a portion of the virtual hospital is presented to the relevant user through extended reality technology.

5. The medical service system according to claim 3, wherein, At least a portion of the virtual hospital is overlaid onto the real-world view of the relevant user using mixed reality technology.

6. The medical service system according to claim 1, wherein, The digital twin includes one or more second digital twins, which can be updated via at least one of the user space applications. The at least one user service is provided through the following methods: Receive an update instruction for at least one of the one or more second digital twins, the update instruction being input by at least one of the relevant users through at least one of the user space applications; as well as The physical entity corresponding to the at least one second digital twin is updated based on the update instruction.

7. The medical service system according to claim 1, wherein, At least a portion of the user services are provided to patients through the patient space application based on the following operations: Based on data related to the patient, detect changes in the patient's medical service process to the target stage; as well as In response to detecting a change in the patient's stage in the medical service process to the target stage, the patient space application performs at least one preset action to provide the patient with at least a portion of the user services.

8. The medical service system according to claim 1, wherein, At least a portion of the user services are provided to doctors through the Medical Space application based on the following operations: Obtain an access request for the medical space application from the doctor's terminal; In response to the access request, the pending tasks to be completed by the doctor are determined based on the time the access request was received and the doctor's schedule information. as well as The doctor's terminal presents an interactive interface through the medical space application. The interactive interface includes interface elements, which are used to obtain at least one auxiliary service related to the task to be processed.

9. The medical service system according to claim 1, wherein, The digital twin maps to the state of the corresponding physical entity and is generated according to a preset data structure. At least a portion of the user services are provided to hospital administrators through an interface provided by the management space application, the interface being provided to the hospital administrators to manage at least a portion of the digital twins.

10. The medical service system according to claim 9, wherein, The preset data structure specifies at least one of the following: the format of the digital twin, the type of information reflected by the digital twin, the storage address of the digital twin, the access point of the digital twin, the update mode of the digital twin, the modification permissions of the digital twin, and the modification permissions of the preset data structure.

11. The medical service system according to claim 1, wherein, The relevant users include a first type of user and a second type of user, and the user space application includes a first user space application corresponding to the first type of user and a second user space application corresponding to the second type of user. To provide user services to relevant users of the physical hospital, the processing device is further configured as follows: Obtain the interaction information between the first type of user and the first user space application; Based on the interaction information, the first user service is provided to the first type of user through the first user space application; as well as Based on the interaction information, the second user service is provided to the second type of user through the second user space application.

12. A healthcare service system, comprising: The hardware devices are configured to collect data related to the physical hospital. The processing equipment is configured as follows: The data is mapped to a virtual hospital corresponding to the physical hospital. The virtual hospital includes an intelligent agent that evolves itself based on the data and artificial intelligence (AI) technology. User services are provided to relevant users of the physical hospital through user space applications, wherein... The user space application is configured to provide the relevant users of the physical hospital with an access point to interact with the intelligent agent. At least a portion of the user services are provided to the relevant user based on the interaction between the relevant user and the agent.

13. The medical service system according to claim 12, wherein, The user space application is configured to present a virtual character corresponding to the agent through extended reality technology, and at least a portion of the user services are provided to the relevant user based on the interaction between the relevant user and the virtual character.

14. The medical service system as described in claim 13, wherein, The virtual character includes a first virtual character corresponding to the registration agent, and the at least part of the user services being provided to the relevant user based on the interaction between the relevant user and the virtual character includes: Obtain the patient's chief complaint through the patient terminal or registration terminal; Based on the patient's chief complaint, at least one candidate department was identified; The content of the first inquiry is determined based on the patient's chief complaint and the at least one candidate department; Control the patient terminal or the registration terminal to display the first virtual avatar, so as to conduct the first inquiry on the patient based on the first inquiry content; and The doctor to whom the patient is registered is determined based on the first data collected by the patient terminal or the registration terminal in the first inquiry.

15. The method as described in claim 14, characterized in that, The virtual character includes a second virtual character corresponding to the pre-diagnosis intelligent agent, and the at least part of the user services provided to the relevant user based on the interaction between the relevant user and the virtual character includes: Based on the doctor's department, determine the content of the second question in the second inquiry; Based on the content of the second inquiry, the patient terminal is controlled to display the second virtual avatar, so as to conduct the second inquiry on the patient based on the content of the second inquiry; and A pre-consultation record is generated based on the second data collected by the patient terminal during the second inquiry.

16. The medical service system according to claim 12, wherein, At least a portion of the user services are provided to the relevant user based on the processing results, which are generated by the agent based on the data.

17. The medical service system according to claim 12, wherein, The data related to the physical hospital includes data related to the medical service process of the physical hospital, and the intelligent agent includes an intelligent agent corresponding to the medical service process. By using the intelligent agent corresponding to the medical service process to process the data related to the medical service process, the user service is provided to the relevant users of the medical service process.

18. The medical service system according to claim 17, wherein, The data related to the medical service process is collected according to a preset data collection protocol corresponding to the medical service process. The preset data collection protocol includes a data collection protocol corresponding to the standard steps of the medical service process.

19. The medical service system according to claim 18, wherein, The processing device is further configured to: In response to the detection of an update to healthcare policies related to the healthcare service process, the preset data collection protocol corresponding to the healthcare service process is updated.

20. The medical service system according to claim 12, wherein, The data related to the medical service process includes data related to patients participating in the medical service process, and the agent corresponding to the medical service process is configured to process the patient-related data by performing the following operations: Monitor updates to the patient-related data; In response to the detection that the patient-related data includes updated data obtained from at least one hardware device, Perform Event of Interest (EOI) detection based on the updated data; as well as In response to the detection of an EOI, one or more preset operations corresponding to the EOI are executed to provide the user service to the relevant user.

21. The medical service system according to claim 20, wherein, The step of performing EOI detection based on the updated data includes: Determine the current stage of the patient's medical service process; Based on the patient's current condition, determine the type of EOI that needs to be detected; and The EOI detection is performed on the updated data based on the type information.

22. The medical service system according to claim 20, wherein, The step of performing EOI detection based on the updated data includes: Based on one or more hardware devices corresponding to the updated data, determine the type information of the EOI to be detected; and The EOI detection is performed on the updated data based on the type information.

23. The medical service system according to claim 20, wherein, The EOI detection is based on EOI detection rules, which are learned by the agent from historical records.

24. The medical service system according to claim 23, wherein, The EOI detection is further performed based on the patient's record data.

25. The medical service system according to claim 20, wherein, One or more preset operations corresponding to the EOI are determined based on the correspondence between the EOI and the preset operations, and the correspondence is learned by the agent from historical records.

26. The medical service system according to claim 25, wherein, One or more preset operations corresponding to the EOI are further determined based on the patient's record data.

27. The medical service system according to claim 12, wherein, At least a portion of the user services are provided to hospital administrators through an interface provided by the management space application, so that the hospital administrators can manage at least a portion of the intelligent agents.

28. The medical service system according to claim 27, wherein, The hospital administrator manages at least a portion of the intelligent agents, including: The basic configuration data of at least a portion of the intelligent agents are presented to the hospital administrators through the aforementioned management space application. Obtain the update information of the basic configuration data input by the hospital administrator through the management space application.

Images