Method and system for remote physical examination

Abstract

The system for remote physical examination may include an operator site and a patient site. The operator site may include a first extended reality device. The patient site may include a physical examination device. The physical examination device may be configured to acquire physical information of the patient, transmit the physical information to the first extended reality device, receive examination information from the extended reality device, and perform a second target examination action on the patient based on the examination information. The first extended reality device may be configured to generate a digital twin model of the patient based on the physical information, display the digital twin model, determine examination information based on a first target examination action performed by the operator based on the digital twin model via the first extended reality device, and transmit the examination information to the physical examination device.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to Chinese Patent Application No. 202310411909.5, filed on April 17, 2023, the contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD The present disclosure relates generally to the medical field, and more particularly to systems and methods for remote physical examinations. [Background technology]

[0003] In real life, due to various reasons, a patient and a doctor may be far away from each other, and the doctor may not be able to perform a physical examination on the patient in person. Therefore, it is desirable to provide a system and method for remote physical examination. Summary of the Invention [Means for solving the problem]

[0004] According to one aspect of the present disclosure, a method for remote physical examination implemented in a physical examination device located at a patient site may include one or more of the following operations: One or more processors may acquire physical information of the patient; The one or more processors may transmit the physical information to a first extended reality device located at an operator site to cause the first extended reality device to generate a digital twin model of the patient based on the physical information; The one or more processors may receive examination information from the first extended reality device; The examination information may be related to a first targeted examination action to be performed by the operator based on the digital twin model via the first extended reality device; and The one or more processors may perform a second targeted examination action on the patient based on the examination information.

[0005] In some embodiments, the physical examination device may include a sensor configured to obtain physical information.

[0006] In some embodiments, the physical examination device may include a first wearable sensory device worn by the patient and configured to perform a physical examination on the patient.

[0007] In some embodiments, the physical examination device may include a mechanical arm configured to perform a physical examination on a patient.

[0008] In some embodiments, the inspection information may include at least one inspection location of the first targeted inspection operation and at least one of a type, a direction, or a first intensity of the first targeted inspection operation.

[0009] In some embodiments, the test information may include at least one test position of the first targeted test operation and a first intensity of the first targeted test operation. Performing a second targeted test operation on the patient based on the test information may include determining whether the first intensity is within a protection range, determining a second intensity based on the protection range in response to determining that the first intensity is outside the protection range, and performing the second targeted test operation on the patient based on the second intensity.

[0010] In some embodiments, the protection range may be determined using a protection range determination model based on at least one of the patient's patient information or medical records.

[0011] In some embodiments, the inspection information may include at least one inspection location of the first target inspection operation and at least one of a type of the first target inspection operation, a direction of the first target inspection operation, or a second intensity corresponding to the at least one inspection location. The second intensity may be determined by the first extended reality device based on the first intensity and the protection range of the first target inspection operation.

[0012] In some embodiments, the digital twin model may include a visualization of at least one affected area of ​​the patient, and the at least one affected area may include at least one of a suspected affected area or a defined affected area.

[0013] In some embodiments, suspected lesions may be predicted using a lesion prediction model based on at least one of the patient's patient information and medical records.

[0014] According to another aspect of the present disclosure, a device for remote physical examination, located at a patient site, may include one or more storage devices and one or more processors configured to communicate with the one or more storage devices. The one or more storage devices may include an instruction set. When the one or more processors execute the instruction set, the one or more processors may be directed to perform one or more of the following operations: The one or more processors may acquire physical information of the patient. The one or more processors may transmit the physical information to a first extended reality device located at an operator site to cause the first extended reality device to generate a digital twin model of the patient based on the physical information. The one or more processors may receive examination information from the first extended reality device. The examination information may be related to a first targeted examination action to be performed by the operator based on the digital twin model via the first extended reality device. The one or more processors may perform a second targeted examination action on the patient based on the examination information.

[0015] According to yet another aspect of the present disclosure, a device for remote physical examination, located at a patient site, may include a first acquisition module configured to acquire physical information of the patient, a first transmission module configured to transmit the physical information to a first extended reality device located at an operator site so that the first extended reality device generates a digital twin model of the patient based on the physical information, a first receiving module configured to receive examination information from the first extended reality device, and an examination module configured to perform a second target examination operation on the patient based on the examination information. The examination information may be related to the first target examination operation to be performed by the operator based on the digital twin model via the first extended reality device.

[0016] According to yet another aspect of the present disclosure, a non-transitory computer-readable medium may include at least one set of instructions. The at least one set of instructions may be executed by one or more processors of a device located at a patient site. The one or more processors may acquire physical information of the patient. The one or more processors may transmit the physical information to a first extended reality device located at an operator site to cause the first extended reality device to generate a digital twin model of the patient based on the physical information. The one or more processors may receive examination information from the first extended reality device. The examination information may be related to a first targeted examination operation performed by the operator based on the digital twin model via the first extended reality device. The one or more processors may perform a second targeted examination operation on the patient based on the examination information.

[0017] According to yet another aspect of the present disclosure, a method for remote physical examination implemented in a first extended reality device located at an operator site may include one or more of the following operations: one or more processors may acquire physical information of a patient; one or more processors may generate a digital twin model of the patient based on the physical information; one or more processors may display the digital twin model; one or more processors may determine examination information based on a first target examination action performed by the operator based on the digital twin model; and one or more processors may transmit the examination information to a physical examination device located at the patient site to cause the physical examination device to perform a second target examination action on the patient based on the examination information.

[0018] In some embodiments, the inspection information may include at least one inspection location of the first targeted inspection operation and at least one of a type, a direction, or a first intensity of the first targeted inspection operation.

[0019] In some embodiments, the inspection information may include at least one inspection location of the first target inspection operation and at least one of a type of the first target inspection operation, a direction of the first target inspection operation, or a second intensity corresponding to the at least one inspection location. The operation of determining the inspection information based on the first target inspection operation may include determining whether the first intensity of the first target inspection operation is within a protection range, and determining a second intensity based on the protection range in response to determining that the first intensity is outside the protection range.

[0020] In some embodiments, the protection range may be determined using a protection range determination model based on at least one of the patient's patient information or medical records.

[0021] In some embodiments, the first extended reality device may include a virtual touch mimicking device worn by at least one hand of the operator, and the one or more processors may simulate tactile sensations experienced by the operator when the first target test action or the second target test action may be performed on the patient based on the test information.

[0022] In some embodiments, the operator may perform a first target inspection operation on the digital twin model.

[0023] In some embodiments, the first extended reality device may include a second wearable sensory device.

[0024] In some embodiments, the operator may perform a first target inspection action on a second wearable sensory device.

[0025] In some embodiments, the digital twin model may include a visualization of at least one affected area of ​​the patient, and the at least one affected area may include at least one of a suspected affected area or a defined affected area.

[0026] In some embodiments, the suspected lesion may be determined using a lesion prediction model based on at least one of the patient's patient information and medical records.

[0027] According to another aspect of the present disclosure, a device for remote physical examination located at an operator site may include one or more storage devices and one or more processors configured to communicate with the one or more storage devices. The one or more storage devices may include an instruction set. When the one or more processors execute the instruction set, the one or more processors may be directed to perform one or more of the following operations: The one or more processors may acquire physical information of a patient; The one or more processors may generate a digital twin model of the patient based on the physical information; The one or more processors may display the digital twin model; The one or more processors may determine examination information based on a first targeted examination action performed by an operator based on the digital twin model; and The one or more processors may transmit the examination information to a physical examination device located at the patient site to cause the physical examination device to perform a second targeted examination action on the patient based on the examination information.

[0028] According to yet another aspect of the present disclosure, a device for remote physical examination provided at an operator side may include a second acquisition module configured to acquire physical information of a patient, a modeling module configured to generate a digital twin model of the patient based on the physical information, a display module configured to display the digital twin model, a third acquisition module configured to determine examination information based on a first target examination action performed by the operator based on the digital twin model, and a second transmission module configured to transmit the examination information to a physical examination device provided at the patient side so that the physical examination device performs a second target examination action on the patient based on the examination information.

[0029] According to yet another aspect of the present disclosure, a non-transitory computer-readable medium may include at least one set of instructions. The at least one set of instructions may be executed by one or more processors of a device located at an operator site. The one or more processors may acquire physical information of a patient. The one or more processors may generate a digital twin model of the patient based on the physical information. The one or more processors may display the digital twin model. The one or more processors may determine examination information based on a first targeted examination action performed by an operator based on the digital twin model. The one or more processors may transmit the examination information to a physical examination device located at the patient site to cause the physical examination device to perform a second targeted examination action on the patient based on the examination information.

[0030] According to yet another aspect of the present disclosure, a system for remote physical examination may include an operator site and a patient site. The operator site may include a first extended reality device. The patient site may include a physical examination device. The physical examination device may be configured to acquire physical information of the patient, transmit the physical information to the first extended reality device, receive examination information from the extended reality device, and perform a second target examination action on the patient based on the examination information. The first extended reality device may be configured to receive the physical information from the physical examination device, generate a digital twin model of the patient based on the physical information, display the digital twin model, determine examination information based on a first target examination action performed by the operator based on the digital twin model via the first extended reality device, and transmit the examination information to the physical examination device.

[0031] Additional features are set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings, or may be learned by the practice or operation of the examples. The features of the disclosure may be realized and attained by practice or use of various aspects of the methods, instrumentalities, and combinations described in the detailed examples set forth below.

[0032] The present disclosure will be further described with reference to exemplary embodiments, which will be described in detail with reference to the drawings, which are non-limiting exemplary embodiments, and in which like reference numerals represent like structure throughout the several views of the drawings. [Brief explanation of the drawings]

[0033] [Figure 1] FIG. 1 illustrates an example block diagram of a system for remote physical examination according to some embodiments of the present disclosure. [Figure 2] 1A-1C are schematic diagrams illustrating front and cross-sectional views of exemplary wearable sensory overalls, according to some embodiments of the present disclosure. [Figure 3A] 1A-1C are schematic diagrams illustrating two exemplary mechanical arms according to some embodiments of the present disclosure. [Figure 3B] 1A-1C are schematic diagrams illustrating two exemplary mechanical arms according to some embodiments of the present disclosure. [Figure 4] FIG. 1 is a schematic block diagram illustrating an exemplary system for patient-site-performed remote physical examination, according to some embodiments of the present disclosure. [Figure 5] FIG. 1 is a schematic block diagram illustrating an exemplary system for operator-site-performed remote physical examination, in accordance with some embodiments of the present disclosure. [Figure 6] 1 is a flowchart illustrating an exemplary process for a patient-site-performed remote physical examination, according to some embodiments of the present disclosure. [Figure 7] 1 is a flowchart illustrating an exemplary process for an operator-site-implemented remote physical examination, according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0034] In the following detailed description, numerous specific details are set forth by way of example to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without such details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level, without detail, to avoid unnecessarily obscuring aspects of the present disclosure. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the claims.

[0035] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" may be intended to include the plural forms unless the context clearly dictates otherwise. It will be further understood that the terms "comprise," "comprises," and / or "comprising," "include," "includes," and / or "including," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0036] It will be understood that the terms "system," "unit," "module," and / or "block" used herein are one way of distinguishing between different components, elements, parts, sections, or assemblies at different levels in ascending order. However, these terms may be substituted by other expressions if they achieve the same purpose.

[0037] In general, the words “module,” “unit,” or “block,” as used herein, refer to logic embodied in hardware or firmware, or a collection of software instructions. The modules, units, or blocks described herein may be implemented as software and / or hardware and stored on any type of non-transitory computer-readable medium or another storage device. In some embodiments, software modules / units / blocks may be compiled and linked into an executable program. It will be appreciated that software modules may be callable from other modules / units / blocks, from themselves, and / or may be invoked in response to detected events or interrupts. Software modules / units / blocks configured for execution on a computing device may be provided on a computer-readable medium, such as a compact disc, digital video disc, flash drive, magnetic disk, or any other tangible medium, or as a digital download (and may initially be stored in a compressed or installable format that requires installation, decompression, or decryption before execution). Such software code may be partially or completely stored on a storage device of the executing computing device for execution by the computing device. Software instructions may also be embedded in firmware, such as an EPROM. It will be further appreciated that the hardware modules / units / blocks may be included in connected logic components such as gates and flip-flops and / or may be included in programmable units such as programmable gate arrays or processors. The modules / units / blocks or computing device functionality described herein may be implemented as software modules / units / blocks, but may also be represented in hardware or firmware.In general, a module / unit / block described herein refers to a logical module / unit / block that may be combined with other modules / units / blocks or divided into sub-modules / sub-units / sub-blocks, regardless of its physical configuration or storage.

[0038] When a unit, engine, module, or block is said to be "on," "connected to," or "coupled to" another unit, engine, module, or block, it is understood that it may be directly on, connected to, or coupled to, or in communication with the other unit, engine, module, or block, or that there may be intervening units, engines, modules, or blocks, unless the context clearly dictates otherwise. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0039] These and other features and characteristics of the present disclosure, as well as the method of operation and function of the associated elements of structure and combination of parts and economies of manufacture, will become more apparent from a consideration of the following description with reference to the accompanying drawings, all of which form a part of this disclosure. It is to be expressly understood, however, that the drawings are for the purposes of illustration and description only and are not intended to limit the scope of the present disclosure. It is understood that the drawings are not to scale.

[0040] The present disclosure provides a method and system for remote physical examination. The method and system rely on extended reality technology to display a digital twin model of a patient to an operator. At the operator side, the operator can comprehensively and intuitively understand the patient's physical condition through the digital twin model and perform a first target physical examination action. Examination information related to the first target physical examination action is transmitted to the patient side, allowing the operator to perform a second target physical examination action on the patient based on the examination information.

[0041] A physical examination can be understood to refer to an examination that achieves the purpose of medical examination by contacting the surface of a human body. For example, a physical examination may include compressing a patient's calf. The patient's feedback regarding the compressing action (e.g., whether it hurts or not) can be used as a reference or basis for a doctor to make a diagnostic judgment. In addition, the person performing the physical examination does not necessarily have to be a doctor, but can be other operators, such as other medical professionals such as technicians, nurses, or physical therapists. The person undergoing the physical examination does not necessarily have to be a patient, but can also be another examination object such as a test subject or a simulated human body. For convenience of explanation, a patient and a doctor will be used as examples in this disclosure.

[0042] The methods and systems for remote physical examinations provided in the present disclosure may be applied to various scenarios. For example, when there is a shortage of local doctors in a certain area, patients in that area may receive a remote physical examination from a doctor in another area. As another example, when a patient has limited mobility or does not want to go out, the patient may receive a remote physical examination from a doctor. As yet another example, the patient may receive a remote physical examination from a doctor when transportation between the patient and the doctor is restricted.

[0043] FIG. 1 is an exemplary block diagram of a system for remote physical examination according to some embodiments of the present disclosure. As shown in FIG. 1, the system 100 may include a physical examination device 110, a first extended reality device 120, and a network 130. The physical examination device 110 may include a first controller and a first external device for the first controller. The first external device may include at least one sensor, a first wearable sensory device, at least one mechanical arm, or the like, or any combination thereof. The first extended reality device 120 may include a second controller and a second external device for the second controller. The second external device may include a head-mounted display device, a touch mimicking device, a second wearable sensory device, or the like, or any combination thereof.

[0044] The physical examination device 110 may belong to the patient side. At least a portion of the patient side (e.g., the first external device) may be considered a local device located at the location where the patient undergoes the remote physical examination. For example, the first external device may be located at the patient's home or a clinic near the patient. The first extended reality device 120 may belong to the operator side. At least a portion of the operator side (e.g., the second external device) may be considered a local device located at the location where the operator performs the remote physical examination. For example, the operator side may be located at a hospital, the operator's home, or a clinic near the operator. In some embodiments, at least a portion of the patient side and / or the operator side may be implemented on a remote server (e.g., a public cloud, a private cloud, or a hybrid cloud). For example, the modeling module 520 of the first extended reality device 120 as shown in FIG. 5 may be implemented on a remote server, and the local head-mounted display device of the first extended reality device 120 may receive the digital twin model generated by the reconstruction module from the remote server.

[0045] The physical examination device 110 may collect physical information of the patient and transmit the physical information to the first extended reality device 120. The patient's physical information may be used to generate a digital twin model of the patient. A digital twin model (digital twin for short) is a computer-generated (e.g., by the first extended reality device 120) virtual projection of a real human body.

[0046] The physical information may include one or more physiological indicators. In some embodiments, the physical information may include body size, posture, body temperature, heart rate, pulse, blood pressure, blood oxygen saturation, complexion, skin smoothness, or odor, or any combination thereof. In some embodiments, the body size may include height, arm span, leg length, shoulder width, or head, chest, waist, or hip circumference, or any combination thereof.

[0047] In some embodiments, the physical examination device 110 may include at least one sensor configured to obtain physical information. For example, with regard to body size / posture, coordinates of multiple preset reference points (e.g., joints) of the patient may be obtained by a position sensor, and the patient's body size / posture may be extracted from the coordinates. In another example, with regard to body posture, orientations of multiple body parts (e.g., upper arms, lower arms, front thighs, and calves) of the patient may be obtained by an angle sensor (e.g., a gyroscope sensor), and the patient's body posture may be extracted from these orientations. In yet another example, physiological indicators of the patient's body temperature, heart rate, pulse, blood pressure, blood oxygen saturation, complexion, or skin smoothness may be obtained by dedicated sensors (e.g., a body temperature sensor, a heart rate sensor, a pulse sensor, a blood pressure sensor, a blood oxygen saturation sensor, a binocular camera, or an odor sensor). In yet another example, a smart stethoscope may be used to obtain sounds from the patient's body (e.g., sounds from organs such as the heart, liver, and lungs, pulse sounds, sounds from blood vessels). A smart stethoscope is a new kind of stethoscope that does not require a physical connection between the operator's ear and the head of the stethoscope. Instead, audio may be transmitted via Bluetooth or other wireless communication channels.

[0048] In some embodiments, at least one sensor may be implemented in a wearable device. For example, various categories of sensors may be implemented in a smart watch to simultaneously measure a patient's temperature, heart rate, pulse, blood pressure, and blood oxygen saturation. In another example, position sensors and pressure sensors may be implemented in wearable sensory overalls.

[0049] The physical examination device 110 may also be configured to perform a physical examination action (eg, a second target examination action) on the patient based on the examination information transmitted from the first extended reality device 120 at the operator side.

[0050] In some embodiments, the physical examination device 110 may include a first wearable sensory device capable of performing a physical examination action on the patient based on the examination information. For example, at least one touch mimicking device may be disposed at one or more locations on the first wearable sensory device to mimic the touch felt by the patient when undergoing the physical examination action. The at least one touch mimicking device may include a vibration component (e.g., a vibration motor), a microcurrent stimulation component, an airbag component, or any combination thereof. The microcurrent stimulation component may have a stimulation intensity within a safe and healthy range for the patient (e.g., 0-500 μA). The airbag component may mimic touch (e.g., the sensation of a finger pressing against the skin) by adjusting the contact area and internal pressure of one or more airbags. As mentioned above, at least one sensor may also be disposed at one or more locations on the first wearable sensory device, and the at least one sensor may collect physical information from the patient. In some embodiments, the touch mimicking device and the sensor at the same location on the first wearable sensory device may be integrated.

[0051] This disclosure is not intended to limit how a wearable sensory device (e.g., the first wearable sensory device and / or the second wearable sensory device referred to below) is designed and implemented. In some embodiments, the wearable sensory device may be tight-fitting overalls. The overalls may cover all or part of the patient's body. Thus, the overalls are convenient for full-body or half-body examinations. In some embodiments, the wearable sensory device may be specialized for localized physical examinations, such as examinations of the head, hands, feet, knees, etc. In these scenarios, the wearable sensory device may be designed as a device that can cover only one body part. For example, the wearable sensory device may be in the form of a helmet, hat, socks, gloves, boots, or knee brace.

[0052] In some embodiments, the first wearable sensory device (e.g., overalls) may also be configured to assist the patient in posing or performing specific movements for clinical purposes (e.g., treating benign paroxysmal positional vertigo). For example, a position sensor and / or angle sensor in the first wearable sensory device may check the patient's posture. A movement sensor (e.g., an acceleration sensor) may check whether the patient is moving or stationary. In some embodiments, when a discrepancy is detected between the patient's posture and a reference posture for treatment or testing, the physical inspection device 110 may assist the patient in correcting their posture based on the discrepancy. For example, the reference posture may be to raise their arm to a reference height. If the patient's arm does not meet the reference height, an airbag component on the patient's arm may apply upward pressure to help the patient raise their arm. When the patient's arm reaches the reference height, the physical inspection device 110 may send a reminder to the patient. For example, a vibration component or microcurrent stimulation component at an arm location of the first wearable sensory device may be controlled to output a stimulation signal (e.g., in the form of a vibration or current) to indicate that the patient has moved to the target location.

[0053] FIG. 2 is a schematic diagram illustrating a front view and a cross-sectional view of an exemplary wearable sensory overall, according to some embodiments of the present disclosure.

[0054] 2, the wearable sensory overalls may be multi-layered. The vibration component 210, the microcurrent stimulation component 220, and the airbag component 230 may be incorporated into three layers, respectively. The airbag component 230 may be incorporated into the innermost layer so that the airbag component 230 may be in direct contact with the surface of the patient's body.

[0055] To ensure effective transmission of touch imitation and accuracy of body measurements, the wearable sensory overalls may be made from a highly elastic material that can be worn tightly on the patient's body.

[0056] In some embodiments, physical examination device 110 may include one or more mechanical arms configured to perform physical examination actions on a patient based on the examination information. In some embodiments, the working ends of the mechanical arms may resemble human hands to perform more realistic physical examination actions. By way of example only, and with reference to FIGS. 3A and 3B , physical examination device 110 may include first mechanical arm 310 and second mechanical arm 320. The two mechanical arms may simulate a face-to-face physical examination performed by an operator with one or both hands on a patient.

[0057] In some embodiments, the patient side may include a positioning device configured to determine positioning information (e.g., positions of multiple points and / or regions) on the surface of the patient's body. The positioning information may be used to generate a digital twin, perform physical examination operations, coordinate transformations, and the like. As mentioned above, in some embodiments, a position sensor on the first wearable sensory device may identify the positions of multiple points on the surface of the patient's body. In other words, the first wearable sensory device has a positioning function (e.g., the first wearable sensory device may be considered a positioning device). In some embodiments, as shown in FIGS. 3A and 3B , the positioning function may be achieved by applying airflow or ultrasound waves within the sensory space 330. The airflow may be emitted by an airflow positioning device. The ultrasound waves may be emitted by an ultrasound positioning device. The positioning device (e.g., an airflow positioning device or an ultrasound positioning device) may be disposed near the patient's body (e.g., at a certain distance from the patient). The positioning device may emit detection waves (ultrasound or airflow) and detect reflected or scattered waves returning from the patient's body surface, thereby locating points and / or regions on the patient's body surface. According to Fig. 3A, when the patient undergoes a physical examination in a standing position within the space 330, the positioning device may be disposed around the patient's body. According to Fig. 3B, when the patient undergoes a physical examination while lying on a bed within the space 330, the positioning device may be disposed above the patient's body.

[0058] The first extended reality device 120 may acquire physical information of the patient, generate a digital twin of the patient based on the physical information, and display the digital twin.

[0059] Here, extended reality is used as a general term for virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0060] VR uses computer technology to create a virtual scene. A user can use a head-mounted display to observe virtual objects in the virtual scene and can use other peripheral devices (also called accessories) to interact with the virtual objects. For example, a user can use VR glasses to observe a virtual user interface, a virtual character, etc. in the virtual scene, and a VR steering wheel to perform input operations on the user interface or interact with the virtual character.

[0061] AR uses computer technology to sense the real world, generate a virtual information layer, and overlay it on the real background visible to the naked eye (for example, displaying vehicle speed and navigation information on a vehicle's windshield), thus completing the combination of reality and virtuality.

[0062] MR uses computer technology to blend the real world with a virtual scene, allowing interaction between the virtual scene and the real world. For example, a virtual character can navigate around a real obstacle when it encounters one. Similar to VR, users can view real objects (such as a real house) and virtual objects (such as a virtual user interface or virtual character) in the blended scene through an MR head-mounted display.

[0063] Through the first extended reality device 120, the operator can view and interact with the patient's digital twin. Depending on the actual technology (VR / AR / MR) used, the display of and / or interaction with the digital twin can be realized through a second external device (such as a head-mounted display or an interactive steering wheel) on the first extended reality device 120, or without a second external device (e.g., interacting with the projection of the digital twin using the naked eye or hands).

[0064] In some embodiments, the first extended reality device 120 may include a touch mimicking device configured to mimic the touch felt by the operator when performing a first target inspection action. The operator may perform the first target inspection action on the digital twin through the touch mimicking device and obtain intuitive feedback from the mimicked touch sensation. The touch mimicking device may be an interactive external device of the first extended reality device 120 (e.g., a VR or MR device) on the operator side. The operator may perform the first target physical inspection action on the digital twin in the virtual scene through the touch mimicking device, and the first extended reality device 120 may capture the first target inspection action and generate corresponding inspection information.

[0065] Touch-mimicking components (e.g., airbag components) and sensors (e.g., position sensors and / or gyroscope sensors) may be disposed at one or more locations on the touch-mimicking device. A detailed description of touch-mimicking components and sensors can be found in the section describing the wearable sensory device.

[0066] The present disclosure does not intend to limit the form of the touch mimicking device. For example, the touch mimicking device may include a touch mimicking glove. In another example, the touch mimicking device may include one or more touch mimicking finger cots. As yet another example, the touch mimicking device may include one or more touch mimicking stickers attached to the pad of at least one finger.

[0067] In some embodiments, system 100 may also include a second extended reality device 140 at the patient site. For details regarding second extended reality device 140, please refer to the section describing first extended reality device 120. First extended reality device 120 and second extended reality device 140 may communicate such that the patient and operator can communicate remotely via first extended reality device 120 and second extended reality device 140. For example, the operator and patient may set up a voice or video call via first extended reality device 120 and second extended reality device 140. In another example, the operator and patient may communicate via text messages via first extended reality device 120 and second extended reality device 140.

[0068] In some embodiments, the patient may request instructions for using the physical examination device 110, such as guidance for wearing the first wearable sensory device or guidance for wearing a wearable device implemented with various sensors, by inputting voice or text into the second extended reality device 140. The second extended reality device 140 may provide instructions (e.g., in the form of audio, video, text, etc.) for using the physical examination device 110 based on the patient's input. When the patient uses the physical examination device 110 incorrectly, the second extended reality device 140 may generate a reminder for the patient.

[0069] In some embodiments, the first extended reality system 120 may include a second wearable sensory device worn by an operator, a third party, or a human body model. For example, the second wearable sensory device may be tight overalls. At least one sensor may be disposed at one or more locations on the second wearable sensory device. In some embodiments, the at least one sensor may include a position sensor and a pressure sensor to detect a test position (e.g., represented by spatial coordinates) on the second wearable sensory device and a force (e.g., including the direction and / or intensity of the force) applied to the test position. The operator may perform a first target test action (P1) at a test position Q1 on the second wearable sensory device. A second controller of the first extended reality device 120 may generate test information and transmit the test information to a first controller of the physical examination device 110. Based on the test information, the patient-side physical examination device 110 may perform a second target test action P2 on the patient via the first wearable sensory device. The test position of the second target test motion performed by the first wearable sensory device may be denoted as Q2. Because the object and the patient wearing the second wearable sensory device may have different body sizes, Q1 needs to be converted to Q2 based on the body size of the object and the body size of the patient, for example, based on the ratio of the body size of the object to the body size of the patient.

[0070] In some embodiments, system 100 may also include an imaging device 150 (e.g., a 360-degree panoramic camera, a 3D camera, etc.) at the patient side. Imaging device 150 may capture real-time images (e.g., two-dimensional (2D) or 3D) of the patient and transmit the real-time images to first extended reality device 120. First extended reality device 120 may display the real-time images.

[0071] Compared to real-time images of a patient, a patient's digital twin may focus on displaying the patient's "hidden" physiological indicators in real time, which may be invisible or barely visible to the naked eye. For example, a digital twin may display the patient's heart rate, temperature, pulse, blood pressure, blood oxygen saturation, smell, and other values ​​in real time.

[0072] Considering the fact that there should not be significant discrepancies between the patient's real-time images and the digital twin, when the operator observes significant discrepancies between the patient's real-time images and the digital twin, it may be an indicator that the system 100 is malfunctioning (e.g., unstable network connection, incorrect generation of the digital twin). In these events, certain measures may be taken, such as improving the quality of the network connection and / or regenerating the digital twin, to ensure the normal functioning of the remote physical examination.

[0073] When an operator and a patient communicate via the first extended reality device 120 and the second extended reality device 140, communication via voice can be a barrier for patients with hearing or speech disabilities, or for patients with accents. In response to the above issues, the second extended reality device 140 may provide a function for displaying subtitles and / or sign language. The patient and the operator may choose whether to enable this function. The first extended reality device 120 and / or the second extended reality device 140 may use a speech recognition model to convert the patient's and / or operator's speech into subtitles for display. The first extended reality device 120 and / or the second extended reality device 140 may use a sign language generation model to convert the operator's speech into sign language for display, and / or may use a sign language generation model to convert the patient's sign language into speech or subtitles for display. This functionality improves communication between the operator and the patient and makes remote physical examinations more efficient.

[0074] In some embodiments, the first extended extension device 120 may use an automatic conference summarization technique or a language model to automatically generate a diagnostic report based on communication between the operator and the patient during the remote physical examination. The diagnostic report may include basic information about the patient, symptoms described by the patient, a diagnosis made by the operator, or treatment recommendations from the operator, or any combination thereof. The operator may not have time to simultaneously record the diagnostic process during the physical examination. Automatically generating a diagnostic report based on communication between the operator and the patient during the physical examination may reduce the operator's workload and pressure.

[0075] Network 130 may be used to facilitate the transmission of information and / or data between the patient site and the operator site. For example, physical examination device 110 and / or imaging device 150 may transmit patient physical information to first extended reality device 120 over network 130. In another example, first extended reality device 120 may transmit examination information to physical examination device 110 over network 130. In yet another example, first extended reality device 120 at the patient site and second extended reality device 140 at the operator site may communicate over network 130.

[0076] For privacy purposes, physical examinations and the collection, storage, and use of personal information (e.g., physical information) must be performed with the patient's consent. In some embodiments, the patient may adjust privacy settings via the second extended reality device 140. For example, with respect to a particular body part, the patient may deny the operator's access for the physical examination and physical information collection. In some embodiments, the operator may request access to the second extended reality device 140 via the first extended reality device 120. In response, the patient may allow or deny access via the second extended reality device 140.

[0077] For safety purposes, system 100 may include a safety module. For example, the force output by a wearable sensory device or mechanical arm may be limited to below a safety threshold. In another example, the current output by a microcurrent stimulation component may also be limited to below a safety threshold. In yet another example, a patient may gradually experience different levels of force or current output from system 100, starting from a minimum level above, until the patient confirms that the force or current currently being experienced is the maximum level they can handle.

[0078] FIG. 4 is a schematic block diagram illustrating an exemplary system for patient-site-performed remote physical examination, according to some embodiments of the present disclosure.

[0079] The system 400 may be implemented in the physical examination device 110 by hardware and / or software components. As shown in FIG. 4 , the patient-side system 400 may include a first acquisition module 410, a first transmission module 420, a first receiving module 430, and an examination module 440.

[0080] The first acquisition module 410 may collect physical information of the patient, details of which can be found in operation 610 of process 600 in FIG.

[0081] The first transmission module 420 may transmit the physical information to the first extended reality device 120 to cause the first extended reality device 120 to generate a digital twin of the patient based on the physical information. Details may be found in operation 620 of process 600 in FIG. 6 .

[0082] The first receiving module 430 may receive the inspection information from the first extended reality device 120. Details may be found in operation 630 of process 600 of FIG.

[0083] The test module 440 may perform a second target test action on the patient based on the test information, details of which can be found in operation 640 of process 600 of FIG.

[0084] 5 is a schematic block diagram illustrating an exemplary system for operator-site-implemented remote physical examination, according to some embodiments of the present disclosure. System 500 may be implemented in first extended reality device 120 by hardware and / or software components.

[0085] As shown in FIG. 5, the system 500 may include a second receiving module 510, a modeling module 520, a display module 530, a third receiving module 540, and a second transmitting module 550.

[0086] The second receiving module 510 may acquire the patient's physical information, details of which may be found in operation 710 of process 700 of FIG.

[0087] The modeling module 520 may generate a digital twin of the patient based on the physical information, details of which may be found in operation 720 of process 700 in FIG.

[0088] The display module 530 may display the digital twin. More details may be found in operation 730 of process 700 in FIG.

[0089] The third receiving module 540 may determine inspection information associated with the first target inspection operation performed by the operator, details of which may be found in operation 740 of process 700 of FIG.

[0090] The second transmission module 550 may transmit the examination information to the physical examination device 110. Details may be found in operation 750 of process 700 of FIG.

[0091] It should be noted that the systems and modules shown in Figures 4 and 5 can be implemented in a variety of ways. For example, the systems and their modules can be implemented by software, hardware, and / or a combination of software and hardware. For example, the hardware portions may be one or more integrated circuits designed for a specific purpose, and the software portions may be stored in one or more storage devices and executed by one or more execution systems, such as microcontrollers or specially designed hardware devices.

[0092] It should be noted that the above system and module descriptions are for convenience only and are not intended to limit the scope of the present disclosure. Those skilled in the art will understand the operating principles of the system and may connect and combine modules, devices, and subsystems in various ways without departing from the operating principles. For example, the first acquisition module 410 and the first output module may be separate modules or may be combined into a single module. In another example, the third receiving module 540 and the second transmitting module 550 may be separate modules or may be combined into a single module. Such variations of the system are also within the scope of the present disclosure.

[0093] FIG. 6 is a flowchart illustrating an exemplary process for a patient-site-performed remote physical examination, according to some embodiments of the present disclosure. In some embodiments, process 600 may be implemented in system 100 shown in FIG. 1. For example, process 600 may be stored in the form of instructions in a storage device and can be invoked and / or executed by physical examination device 110 (e.g., a first controller of physical examination device 110, or one or more modules shown in FIG. 4). The operations of illustrated process 600 presented below are intended to be exemplary. In some embodiments, process 600 may be achieved with one or more additional operations not described and / or without one or more of the operations described. Furthermore, the order of operations of process 600 shown in FIG. 6 and described below is not intended to be limiting.

[0094] At 610, the physical examination device 110 (eg, the first acquisition module 410) may acquire physical information of the patient.

[0095] Further information regarding methods for acquiring physical information may be found elsewhere in this disclosure (e.g., in connection with FIG. 1). For example, the first acquisition module 410 may include at least one sensor configured to acquire the physical information shown in FIG. 1. As another example, the first acquisition module 410 may acquire the physical information from at least one sensor shown in FIG. 1.

[0096] At 620, physical examination device 110 (e.g., first transmission module 420) may transmit the physical information to first extended reality device 120 to cause first extended reality device 120 to generate a digital twin of the patient based on the physical information. More details regarding generating a digital twin by first extended reality device 120 may be found elsewhere in this disclosure (e.g., in connection with FIG. 7).

[0097] At 630, physical examination device 110 (e.g., first receiving module 430) may receive examination information from first extended reality device 120. The examination information may be related to a first target examination operation performed by an operator (e.g., a physician) based on the digital twin via first extended reality device 120. More details regarding determining the examination information by first extended reality device 120 may be found elsewhere in this disclosure (e.g., in connection with FIG. 7, and particularly with operation 740 of process 700 of FIG. 7).

[0098] At 640, the physical examination device 110 (e.g., the examination module 440) may perform a second target examination action on the patient based on the examination information. In some embodiments, the first target examination action may be the same as or different from the second target examination action. In some embodiments, the examination module 440 may include a first wearable sensory device and / or at least one mechanical arm.

[0099] In some embodiments, the inspection information represents a first target inspection operation and may include at least one inspection position of the first target inspection operation and at least one of an inspection type, an inspection direction, or a first intensity of the first target inspection operation. In some embodiments, the inspection position may include a label indicating a body part and / or a spatial coordinate of a position on the patient's body surface. For example, if the remote physical examination requires a lower level of spatial accuracy, the inspection position in the inspection information may be a body part label. In another example, if the remote physical examination requires a high level of spatial accuracy, the inspection position in the inspection information may include a spatial coordinate of a position on the patient's body surface. In some embodiments, the inspection type may include one or more of pressing, stroking, or tapping. In some embodiments, the first intensity of the first target inspection operation may be represented by a level (e.g., low / medium / high). In some embodiments, the first intensity of the first target inspection operation may be represented by a value. In some embodiments, the inspection direction of the first target inspection operation refers to the direction of the force applied to at least one inspection position in the first target inspection operation. Note that the inspection direction may be omitted from the inspection information. For example, the system 100 may automatically determine that the inspection direction is perpendicular to the body surface of the corresponding inspection position.

[0100] In some embodiments, the physical examination device 110 may restore a first target examination action for the patient based on the examination information. Restoring a first target examination action for the patient means that the physical examination device 110 may perform a second target examination action that is the same as the first target examination action. For example, if the examination information includes "right knee" (examination position) and "tap" (examination type), the second target examination action may be tapping the right knee. As another example, the inspection information may be represented as a group of parameters characterizing a first target inspection operation, such as (S1, L1, F1, S2, L2, F2), where S1 represents a first inspection position, L1 represents a first inspection direction of the force applied to the first inspection position S1, F1 represents a value of the strength of the force applied to the first inspection position S1, S2 represents a second inspection position, L2 represents a second inspection direction of the force applied to the second inspection position S2, and F2 represents a value of the strength of the force applied to the second inspection position S2. The physical examination device 110 may generate parameters characterizing a first target examination action based on the examination information and restore the first target examination action for the patient based on the parameters, for example, the physical examination device 110 may control the first wearable sensory device and / or at least one mechanical arm to move to restore the first target examination action for the patient based on the parameters.

[0101] In some embodiments, the physical inspection device 110 may control an airbag component disposed at a first inspection location S1 on the first wearable sensory device to apply a force having a first inspection direction L1 and a value F1, and may control an airbag component disposed at a second inspection location S2 on the first wearable sensory device to apply a force having a second inspection direction L2 and a value F2. In some embodiments, the physical inspection device 110 may control a mechanical arm to apply a force having a first inspection direction L1 and a value F1 to the first inspection location S1 on the body surface, and to apply a force having a second inspection direction L2 and a value F2 to the second inspection location S2 on the body surface.

[0102] In some embodiments, the ultrasonic positioning device and the airflow positioning device may also have a touch imitation function. Correspondingly, the ultrasonic positioning device and / or the airflow positioning device may also be used to perform a second target inspection operation on the patient. Like airflow, ultrasound can also be sensed by the human body. After placing the ultrasonic positioning device and / or the airflow positioning device in an enclosed environment (such as an enclosed indoor space), accurate positioning and touch imitation on the patient's body surface can be achieved by adjusting the parameters of the emitted waves from the ultrasonic positioning device and / or the airflow positioning device. Taking the airflow positioning device as an example, the airflow positioning device may include an array of air outlets, and the physical inspection device 110 may control parameters of the airflow output by the air outlets, such as the airflow direction, the airflow intensity (speed), the shape of the air outlets, and the duration of outputting the airflow, to achieve accurate positioning and touch imitation on the patient's body surface. Because the positioning device (e.g., an ultrasound positioning device and / or an airflow positioning device) has a positioning function, the positioning device can accurately reconstruct the test position to perform the second target test operation. Also, similar to the example in the previous paragraph, when the test information is (S1, L1, F1, S2, L2, F2), the physical examination device 110 may adjust parameters of the emitted wave (e.g., ultrasound or airflow) so that a force having a first direction L1 and a value F1 is applied to the first test position S1 on the patient's body surface, and a force having a second direction L2 and a value F2 is applied to the second test position S2 on the patient's body surface.

[0103] When the operator performs the first target inspection motion via the first extended reality device 120, the force of the first target inspection motion may be too large because the first target inspection motion does not directly act on the patient. If the force of the first target inspection motion is too large with respect to trauma or a force-sensitive location on the patient, mapping the first target inspection motion to the mechanical arm or the first wearable sensory device may result in secondary trauma or discomfort to the patient.

[0104] To solve the above problem, the physical testing device 110 may determine whether the first intensity is within a protection range. In response to determining that the first intensity is outside the protection range, the physical testing device 110 may determine a second intensity based on the protection range. The physical testing device 110 may perform a second target test operation on the patient based on the second intensity. In response to determining that the first intensity is within the protection range, the physical testing device 110 may restore the first target test operation on the patient based on the test information, for example, the physical testing device 110 may perform a second target test operation that is the same as the first target test operation.

[0105] In some embodiments, the protection range may be determined using a protection range determination model based on at least one of the patient's patient information or medical records. The protection range determination model may be a machine learning model. An input of the protection range determination model may include at least one of the patient's patient information or medical records. An output of the protection range determination model may include a protection range. In some embodiments, the patient may correspond to a single protection range. In some embodiments, different parts of the patient may correspond to different protection ranges. For example, the patient's legs may correspond to a first protection range and the patient's breasts may correspond to a second protection range.

[0106] The patient's patient information may include age, sex, or occupation, etc., or any combination thereof. The patient's medical records may include the type of disease, the location of the disease, one or more medical images of the patient, or treatment information for the disease, etc., or any combination thereof.

[0107] In some embodiments, the protection range determination model may be provided by training a preliminary model using sample data. The sample data may include a plurality of sample datasets. Each of the plurality of sample datasets may include training data and label data of a sample patient. The training data of the sample patient may include patient information and / or medical records related to an in-person physical examination of the sample patient. The label data of the sample patient may include the protection range of the sample patient. The protection range of the sample patient may be obtained based on feedback from the sample patient during the in-person physical examination.

[0108] In some embodiments, in response to determining that the first intensity is less than the minimum value of the protection range, physical inspection device 110 may determine the minimum value of the protection range as the second intensity. In response to determining that the first intensity is greater than the maximum value of the protection range, physical inspection device 110 may determine the maximum value of the protection range as the second intensity.

[0109] In some embodiments, the process for adjusting the first intensity to the second intensity based on the protection range may be performed by first extended reality device 120. In this case, the examination information may include at least one examination position and at least one of an examination type, an examination direction, and a second intensity. Physical examination device 110 may directly perform a second target examination action on the patient according to the examination information. The process for adjusting the first intensity to the second intensity based on the protection range performed by first extended reality device 120 may be similar to the above-described process for adjusting the first intensity to the second intensity based on the protection range performed by physical examination device 110.

[0110] In some embodiments, after the physical testing device 110 performs the second target test motion on the patient, the physical testing device 110 may record the patient's feedback (e.g., video, audio, text, etc.) regarding the second target test motion and adjust the protection range based on the patient's feedback. For example, after the physical testing device 110 performs the second target test motion on the patient, the patient may indicate that they cannot tolerate the force of the second target test motion. The physical testing device 110 may identify the patient's audio indication and reduce the maximum protection value.

[0111] In some embodiments, spatial coordinates of an inspection position of a first target inspection operation performed via the first extended reality device 120 (e.g., the second wearable sensory device or the digital twin) may be transformed into spatial coordinates of a corresponding position of the patient. The coordinate transformation may be performed by the first extended reality device 120 or the physical inspection device 110. The transformation relationship may be determined based on the coordinate system of the first extended reality device 120 and the coordinate system of the physical inspection device 110. Furthermore, the transformation relationship may be determined based on the ratio between the size of the digital twin and the size of the patient's body, or the ratio between the size of the object wearing the second wearable sensory device and the size of the patient's body.

[0112] It should be noted that the above description is provided for illustrative purposes only and is not intended to limit the scope of the present disclosure. Those skilled in the art may make many variations and modifications under the teachings of the present disclosure. However, these variations and modifications do not depart from the scope of the present disclosure.

[0113] FIG. 7 is a flowchart illustrating an exemplary process for operator-site-implemented remote physical examination, according to some embodiments of the present disclosure. In some embodiments, process 700 may be implemented in system 100 shown in FIG. 1. For example, process 700 may be stored in the form of instructions in a storage device and can be invoked and / or executed by first extended reality device 120 (e.g., a second controller of first extended reality device 120, or one or more modules shown in FIG. 5). The operations of illustrated process 700 presented below are intended to be exemplary. In some embodiments, process 700 may be accomplished with one or more additional operations not described and / or without one or more of the operations described. Furthermore, the order of operations of process 700 shown in FIG. 7 and described below is not intended to be limiting.

[0114] At 710, the first extended reality device 120 (eg, the second receiving module 510) may acquire physical information of the patient.

[0115] More details regarding physical information may be found elsewhere in this disclosure (e.g., in connection with FIGS. 1 and 6). In some embodiments, at least some of the physical information may be obtained from the patient. For example, when a patient comes for a follow-up visit, the first extended reality device 120 may retrieve the patient's static physical information from the patient's medical record. Alternatively, the patient may inform the operator of their age, gender, and body size in a conversation. Static physical information refers to physical information that remains constant over a long period of time (e.g., within one year) or that remains at least constant during the visit. Static physical information may include age, gender, or body size, or any combination thereof.

[0116] At 720, the first extended reality device 120 (e.g., the modeling module 520) may generate a digital twin of the patient based on the body information.

[0117] At 730, the first extended reality device 120 (e.g., display module 530) may display the digital twin model. In some embodiments, the first extended reality device 120 may display the digital twin model and real-time images (e.g., 2D or 3D) of the patient together or separately.

[0118] A digital twin may include a three-dimensional (3D) body model and annotation information.

[0119] The 3D body model may be generated based on the patient's body size and / or real-time video of the patient. The 3D body model may include a body surface. The body surface may refer to naked parts (skin, nails, etc.) and semi-nude parts (nostrils, etc.) of the human body. In some embodiments, the 3D body model may also include internal structures of the human body (bones, organs, tissues, blood vessels, etc.). In some embodiments, the first extended reality device 120 may calibrate a reference human body model based on the patient's body size and / or real-time video to obtain the 3D body model of the patient.

[0120] The annotation information may include patient body information. For example, the first extended reality device 120 may label the patient's heart rate in a region of the heart of the 3D body model. In another example, the first extended reality device 120 may label the skin color and smoothness in a target skin region of the 3D body model. As yet another example, the first extended reality device 120 may label the patient's current blood pressure on the 3D body model. In some embodiments, the annotation information may also include data from the patient's past diagnoses. For example, the first extended reality device 120 may label the location and / or type of the patient's abnormality (e.g., trauma, tumor, etc.). The annotation information may provide reference for the operator to perform the first target examination operation.

[0121] In some embodiments, the operator may zoom in or out on the 3D body model. Of course, zooming in / out only changes the size of the 3D body model, not its shape.

[0122] In some embodiments, the first extended reality device 120 may include an audio output device (such as earphones) that may play sounds collected from the patient's body (e.g., heart sounds, lung sounds, etc.) from a patient-side smart stethoscope for the operator's reference.

[0123] In some embodiments, the first extended reality device 120 may generate a real-time digital twin of the patient based on the body information and display the digital twin model in real time. For example, when the patient performs an exercise (e.g., the patient's posture and / or position changes), the digital twin may perform a corresponding exercise. As another example, annotation information of the digital twin (e.g., the patient's body information) may be displayed in real time. In some embodiments, the digital twin may present a fixed posture and / or position of the patient that does not change as the patient moves. For example, the first extended reality device 120 may reconstruct a 3D body model of the digital twin based on at least one video frame in which the patient is in a standing posture, resulting in a digital twin in a standing posture. When the patient is sitting in a chair, the digital twin may still maintain a standing posture.

[0124] In some embodiments, the digital twin may include a visualization of at least one affected area of ​​the patient, which may include at least one of a suspected affected area and a defined affected area.

[0125] The first extended reality device 120 may reconstruct the suspected lesion based on medical records, real-time video, or one or more medical images of the patient, or any combination thereof.

[0126] The suspected lesion may be predicted using a lesion prediction model based on at least one of patient information or medical records of the patient. The lesion prediction model may be a machine learning model. Inputs of the lesion prediction model may include at least one of patient information or medical records of the patient. Outputs of the lesion prediction model may include the suspected lesion.

[0127] In some embodiments, the lesion prediction model may be provided by training a preliminary model using sample data. The sample data may include a plurality of sample datasets. Each of the plurality of sample datasets may include training data and label data for a sample patient. The training data for the sample patient may include patient information and / or medical records associated with a physician visit for the sample patient. The label data for the sample patient may include a lesion determined at a subsequent physician visit rather than at the current physician visit.

[0128] Through the digital twin, the operator may directly view the suspected and / or defined lesions, which allows the operator to more accurately diagnose the patient and perform more appropriate first target examination actions by referencing the suspected and / or defined lesions.

[0129] At 740, the first extended reality device 120 (e.g., the third receiving module 540) may determine inspection information based on a first target inspection action performed by the operator based on the digital twin.

[0130] In some embodiments, the operator may use a touch-imitating device (such as a touch-imitating glove) to perform a first target inspection operation by touching the digital twin or the real-time image, and the first extended reality device 120 may detect an inspection location and at least one of an inspection type, an inspection direction, and a first intensity of the first target inspection operation. In some embodiments, the first extended reality device 120 may determine at least one of an inspection location and at least one of an inspection type, an inspection direction, and a first intensity of the first target inspection operation as inspection information. For illustrative purposes, descriptions related to performing the first target inspection operation may use a digital twin as an example. The operation of performing the first target inspection operation on the real-time image may be similar to the operation of performing the first target inspection operation on the digital twin.

[0131] In some embodiments, in response to detecting that the touch mimicking device has touched the digital twin (e.g., a point on the touch mimicking device has overlapped with any point on the digital twin), the first extended reality device 120 (e.g., a VR or MR device) may seek confirmation from the operator regarding whether to generate test information. In response to determining that the operator has confirmed that generation of test information is necessary, the first extended reality device 120 may further seek confirmation regarding details of the test information to be generated. For example, the first extended reality device 120 may seek confirmation regarding whether to generate a type, direction, or intensity of a first target test motion. The first extended reality device 120 may obtain the patient's confirmation through the patient's audio, video, and / or text input. In some embodiments, the operator may input at least a portion of the test information (e.g., at least one of the test position, test type, test direction, and test intensity) by voice or text. In some embodiments, the operator may touch the digital twin with one hand to determine at least one inspection location and input at least one of an inspection type, an inspection direction, and a first intensity with the other hand. In some embodiments, in response to determining that the operator confirms that generation of inspection information is required, first extended reality device 120 may directly generate the inspection information. Information regarding whether to generate a type, direction, or intensity of a first target inspection motion and / or which type, direction, or intensity of the first target inspection motion should be included in the inspection information may be default for first extended reality device 120 or may be preset by the operator. In some embodiments, in response to detecting that a touch mimicking device has touched the digital twin, first extended reality device 120 may directly generate the inspection information.Information regarding whether to generate a type, direction, or intensity of the first target inspection motion and / or which type, direction, or intensity of the first target inspection motion should be included in the inspection information may be a default for the first extended reality device 120 or may be pre-set by an operator.

[0132] In some embodiments, the touch mimicking device (e.g., a touch mimicking glove) may output a force back to the operator, which may have the same value as the first intensity, so that the operator may confirm whether the input first intensity is valid. When the operator confirms that the input first intensity is valid, the first extended reality device 120 may generate test information based on at least one test location touched by the touch mimicking device and the first intensity confirmed as valid by the operator.

[0133] In some embodiments, the first extended reality device 120 may use the testing information to mimic the touch felt by the operator when performing a first target testing action on the patient via a touch mimicking device. For example, the touch mimicking device may output a force to the operator having a magnitude equal to the first magnitude and a direction opposite to the testing direction.

[0134] In some embodiments, in response to detecting that the operator's hand has contacted the digital twin (e.g., a point on the hand has overlapped with any point on the digital twin), the first extended reality device 120 (e.g., an MR device) may seek confirmation from the operator regarding whether to generate test information. In response to determining that the operator has confirmed generating test information, the first extended reality device 120 may seek confirmation regarding the value of the force to be applied. A method of confirming the force value may include obtaining a value input by the operator. In some embodiments, the operator may input the force value by audio input, or the operator may use one hand to select at least one test location and the other hand to input the force value. In some embodiments, the first extended reality device 120 may determine the test direction of the force by detecting the posture of the operator's hand.

[0135] In some embodiments, in response to detecting that an operator has performed a first target inspection action on a second wearable sensory device (e.g., wearable sensory overalls), corresponding inspection information may be acquired. The second wearable sensory device may capture the first target inspection action and generate the corresponding inspection information. Further details regarding capturing the first target inspection action via the second wearable sensory device may be found in the description of FIG. 1.

[0136] In some embodiments, the operator may directly input the inspection information without performing the first target inspection action, for example, the operator may manually or vocally input the inspection information.

[0137] In some embodiments, the process for adjusting the first intensity to the second intensity based on the protection range may be performed by first extended reality device 120. In this case, the examination information may include at least one examination location and at least one of an examination type, an examination direction, and a second intensity. The process for adjusting the first intensity to the second intensity based on the protection range performed by first extended reality device 120 may be similar to the process for adjusting the first intensity to the second intensity based on the protection range performed by physical examination device 110 (e.g., as shown in FIG. 6, and particularly operation 640 of process 600 of FIG. 6).

[0138] At 750, the first extended reality device 120 (e.g., second transmitting module 550) may transmit the examination information to the physical examination device 110 to cause the physical examination device 110 to perform a second targeted examination action on the patient based on the examination information. Details regarding performing a second targeted examination action on the patient based on the examination information by the physical examination device 110 may be found elsewhere in this disclosure (e.g., in connection with FIG. 6, and particularly with respect to operation 640 of process 600 of FIG. 6).

[0139] In some embodiments, spatial coordinates of an inspection position of a first target inspection operation performed via the first extended reality device 120 (e.g., the second wearable sensory device or the digital twin) may be transformed into spatial coordinates of a corresponding position of the patient. The coordinate transformation may be performed by the first extended reality device 120 or the physical inspection device 110. The transformation relationship may be determined based on the coordinate system of the first extended reality device 120 and the coordinate system of the physical inspection device 110. Furthermore, the transformation relationship may be determined based on the ratio between the size of the digital twin and the size of the patient's body, or the ratio between the size of the object wearing the second wearable sensory device and the size of the patient's body.

[0140] The beneficial effects that may be brought about by the embodiments of the present disclosure include, but are not limited to, the following: (1) Using the patient's digital twin as a medium to realize remote physical examination, the operator can comprehensively and intuitively understand the patient's true physical condition through the digital twin; (2) Through the touch mimicking device, the operator can experience the realistic touch of physical examination.

[0141] Having thus described the basic concepts, it will be fairly clear to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Although not expressly stated herein, various changes, improvements, and modifications may be made and are contemplated by those skilled in the art. These changes, improvements, and modifications are intended to be suggested by this disclosure and are within the spirit and scope of the exemplary embodiments of the present disclosure.

[0142] Furthermore, certain terms are used to describe embodiments of the present disclosure. For example, the terms "one embodiment," "embodiments," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure. Therefore, it should be emphasized and recognized that two or more references to "an embodiment" or "one embodiment" or "alternative embodiments" in various parts of this specification do not necessarily all refer to the same embodiment. Furthermore, certain features, structures, or characteristics may be combined as appropriate in one or more embodiments of the present disclosure.

[0143] Moreover, it will be recognized by those skilled in the art that aspects of the present disclosure may be shown and described herein in any of many patentable classes or contexts, including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be embodied in entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or combinations of software and hardware implementations, all of which may be referred to generally herein as "units," "modules," or "systems." Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer-readable medium(s) having computer-readable program code embodied thereon.

[0144] A computer-readable signal medium may include a propagated data signal having computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including an electromagnetic signal, an optical signal, or the like, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium, other than a computer-readable storage medium, that may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied in a computer-readable signal medium may be transmitted using any suitable medium, including wireless, wired, fiber optic cable, RF, or the like, or any suitable combination of the foregoing.

[0145] Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, or Python; the "C" programming language; traditional procedural programming languages ​​such as Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider), or in a cloud computing environment, or may be provided as a service, such as Software as a Service (SaaS).

[0146] Furthermore, the described order of processing elements or sequences, or the use of numbers, letters, or other designations, therefore, are not intended to limit the claimed processes and methods to any order, unless expressly stated in the claims. While the above disclosure sets forth, through various examples, what are presently believed to be various useful embodiments of the present disclosure, it should be understood that such details are for the purpose only, and that the appended claims are not limited to the disclosed embodiments, but on the contrary, are intended to cover modifications and equivalent arrangements within the spirit and scope of the disclosed embodiments. For example, implementations of the various components described above may be implemented in hardware devices, but may also be implemented as software-only solutions, e.g., installation on existing servers or mobile devices.

[0147] Similarly, in the foregoing description of embodiments of the present disclosure, it should be appreciated that various features may be grouped together in a single embodiment, figure, or description for the purpose of streamlining the disclosure to aid in understanding one or more of the various embodiments. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, claimed subject matter may comprise fewer than all features of a single foregoing disclosed embodiment. [Explanation of symbols]

[0148] 100 systems 110 Physical Examination Devices 120 First Extended Reality Device 130 Network 140 Second Extended Reality Device 150 Imaging Device 200 Wearable Sensory Overalls 210 Vibration Components 220 Microcurrent stimulation components 230 Airbag Components 310 First Mechanical Arm 320 Second Mechanical Arm 330 Sensory Space 400 System 410 First Acquisition Module 420 first transmitting module 430 first receiving module 440 Inspection Module 500 Systems 510 second receiving module 520 Modeling Module 530 Display Module 540 Third Receiving Module 550 Second Transmitting Module

Claims

1. 1. A method for remote physical examination, implemented in a patient-site physical examination device, the method comprising: acquiring physical information of a patient; transmitting the physical information to a first extended reality device located at an operator site, such that the first extended reality device generates a digital twin model of the patient based on the physical information; receiving inspection information from the first extended reality device, the inspection information relating to a first target inspection action to be performed by an operator based on the digital twin model via the first extended reality device; performing a second target examination action on the patient based on the examination information; A method comprising:

2. The method of claim 1 , wherein the physical examination device includes a sensor configured to obtain the physical information.

3. 3. The method of claim 1 or 2, wherein the physical examination device includes a first wearable sensory device worn by the patient and configured to perform the physical examination on the patient.

4. The method of claim 1 , wherein the physical examination device comprises a mechanical arm configured to perform the physical examination on the patient.

5. 5. The method of claim 1, wherein the inspection information includes at least one inspection position of the first target inspection operation and at least one of a type, a direction, or a first intensity of the first target inspection operation.

6. the inspection information includes at least one inspection position of the first target inspection operation and a first intensity of the first target inspection operation; The step of performing the second target examination action on the patient based on the examination information includes: determining whether the first intensity is within a protection range; responsive to determining the first intensity to be outside the protection range, determining a second intensity based on the protection range; performing the second target examination action on the patient based on the second intensity; 6. The method of any one of claims 1 to 5, comprising:

7. The method of claim 6 , wherein the protection scope is determined using a protection scope determination model based on at least one of patient information or medical records of the patient.

8. the inspection information includes at least one inspection position of the first target inspection operation and at least one of a type of the first target inspection operation, a direction of the first target inspection operation, or a second intensity corresponding to the at least one inspection position; the second intensity is determined by the first extended reality device based on the first intensity and a protection range of the first target inspection motion.

5. The method according to any one of claims 1 to 4.

9. 9. The method of claim 1, wherein the digital twin model includes a visualization of at least one affected area of ​​the patient, the at least one affected area including at least one of a suspected affected area or a defined affected area.

10. 10. The method of claim 9, wherein the suspected lesion is predicted using a lesion prediction model based on at least one of patient information or medical records of the patient.

11. A patient-site-based device for remote physical examination, comprising: at least one storage device containing an instruction set; at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor: Obtaining patient physical information; transmitting the physical information to a first extended reality device located at an operator site, such that the first extended reality device generates a digital twin model of the patient based on the physical information; receiving inspection information from the first extended reality device, the inspection information relating to a first target inspection action to be performed by an operator based on the digital twin model via the first extended reality device; and performing a second target examination action on the patient based on the examination information; at least one processor instructed to perform operations including: A device comprising:

12. A patient-site-based device for remote physical examination, comprising: a first acquisition module configured to acquire physical information of the patient; a first transmitting module configured to transmit the physical information to a first extended reality device at an operator site so that the first extended reality device generates a digital twin model of the patient based on the physical information; a first receiving module configured to receive inspection information from the first extended reality device, the inspection information relating to a first target inspection action to be performed by an operator based on the digital twin model via the first extended reality device; an examination module configured to perform a second target examination action on the patient based on the examination information; A device comprising:

13. 1. A non-transitory computer-readable medium comprising at least one set of instructions for remote physical examination, the at least one set of instructions, when executed by one or more processors of a patient-located device, causing the device to perform a method, the method comprising: acquiring physical information of a patient; transmitting the physical information to a first extended reality device located at an operator site, such that the first extended reality device generates a digital twin model of the patient based on the physical information; receiving inspection information from the first extended reality device, the inspection information relating to a first target inspection action to be performed by an operator based on the digital twin model via the first extended reality device; performing a second target examination action on the patient based on the examination information; 1. A non-transitory computer-readable medium comprising:

14. 1. A method for remote physical examination implemented in a first extended reality device located at an operator's location, the method comprising: acquiring physical information of a patient; generating a digital twin model of the patient based on the physical information; displaying the digital twin model; determining inspection information based on a first target inspection action performed by an operator based on the digital twin model; transmitting the test information to a physical examination device at a patient site so that the physical examination device performs a second target test action on the patient based on the test information; A method comprising:

15. The method of claim 14 , wherein the inspection information includes at least one inspection position of the first target inspection operation and at least one of a type, a direction, or a first intensity of the first target inspection operation.

16. the inspection information includes at least one inspection position of the first target inspection operation and at least one of a type of the first target inspection operation, a direction of the first target inspection operation, or a second intensity corresponding to the at least one inspection position; The step of determining the inspection information based on the first target inspection operation includes: determining whether a first intensity of the first target inspection motion is within a protection range; determining the second intensity based on the protection range in response to determining that the first intensity is outside the protection range; and 15. The method of claim 14, comprising:

17. The method of claim 16 , wherein the protection scope is determined using a protection scope determination model based on at least one of patient information or medical records of the patient.

18. the first extended reality device includes a virtual touch mimicking device worn by at least one hand of the operator; The method comprises: a step of simulating a tactile sensation obtained by the operator when the first target inspection action or the second target inspection action is performed on the patient based on the inspection information; further comprising:

18. The method of any one of claims 14 to 17.

19. 19. The method of claim 14, wherein the operator performs the first target inspection operation on the digital twin model.

20. 20. The method of claim 14, wherein the first extended reality device includes a second wearable sensory device.

21. 21. The method of claim 20, wherein the operator performs the first target inspection action on the second wearable sensory device.

22. 22. The method of any one of claims 14 to 21, wherein the digital twin model includes a visualization of at least one affected area of ​​the patient, the at least one affected area including at least one of a suspected affected area or a defined affected area.

23. 23. The method of claim 22, wherein the suspected lesion is determined using a lesion prediction model based on at least one of patient information or medical records of the patient.

24. A device for remote physical examination, located at an operator's site, comprising: at least one storage device containing an instruction set; at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor: Obtaining patient physical information; generating a digital twin model of the patient based on the physical information; displaying the digital twin model; determining inspection information based on a first target inspection action performed by an operator based on the digital twin model; and transmitting the test information to a physical examination device located at a patient site so that the physical examination device performs a second target test action on the patient based on the test information; at least one processor instructed to perform operations including: A device comprising:

25. A device for remote physical examination, located at an operator's site, comprising: a second acquisition module configured to acquire physical information of the patient; a modeling module configured to generate a digital twin model of the patient based on the physical information; and a display module configured to display the digital twin model; and a third acquisition module configured to determine inspection information based on a first target inspection action performed by an operator based on the digital twin model; a second transmitting module configured to transmit the test information to a physical examination device located at a patient site so that the physical examination device performs a second target test action on the patient based on the test information; A device comprising:

26. 1. A non-transitory computer-readable medium comprising at least one set of instructions for remote physical examination, the at least one set of instructions, when executed by one or more processors of a device located at an operator site, causing the device to perform a method, the method comprising: acquiring physical information of a patient; generating a digital twin model of the patient based on the physical information; displaying the digital twin model; determining inspection information based on a first target inspection action performed by an operator based on the digital twin model; transmitting the test information to a physical examination device at a patient site so that the physical examination device performs a second target test action on the patient based on the test information; 1. A non-transitory computer-readable medium comprising:

27. 1. A system for remote physical examination, comprising an operator site and a patient site, the operator site including a first extended reality device and the patient site including a physical examination device; The physical examination device comprises: Obtaining the patient's physical information; transmitting the physical information to the first extended reality device; receiving inspection information from the extended reality device; performing a second target examination action on the patient based on the examination information; It is configured as follows: the first extended reality device receiving the physical information from the physical examination device; generating a digital twin model of the patient based on the physical information; Displaying the digital twin model; determining the inspection information based on a first target inspection action performed by an operator based on the digital twin model via the first extended reality device; Transmitting the examination information to the physical examination device The system is configured as follows:

Images