Doctor-patient communication system

Through augmented reality technology, the surgical simulation was performed using a three-dimensional reconstruction model before the operation, which solved the problem of communication disorders between doctors and patients, improved the understanding and coordination before the operation, and provided postoperative rehabilitation suggestions, achieving efficient doctor-patient communication.

CN119069145APending Publication Date: 2024-12-03WUHAN UNITED IMAGING HEALTHCARE SURGICAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310645230.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Before difficult surgery, inconsistent medical cognition between doctors and patients leads to communication disorders. The existing visual lesions display method is not intuitive enough, and it is difficult for patients to understand the surgical process, which affects the communication effect.

Method used

Using augmented reality (AR) technology, the doctor's AR operation terminal and the patient's AR operation terminal are used to simulate the operation using the three-dimensional reconstruction of the preoperative part AR model generated by the surgical planning workstation, and display it simultaneously on the doctor's and patients' terminals to realize virtual meetings. Data sharing and editing in the room.

Benefits of technology

Through intuitive AR technology, the communication complexity is reduced, the patient's understanding and coordination of the surgery is improved, the communication effect is enhanced, and the AR visualization function is provided for postoperative rehabilitation suggestions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119069145A_ABST
    Figure CN119069145A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of doctor-patient communication processing, provides a doctor-patient communication system, and aims to visually display an operation process to a patient by using an AR (Augmented Reality) technology. In the doctor-patient communication system, a doctor AR operation terminal is used for obtaining a preoperative part AR model of a patient from an operation planning workstation in response to a doctor instruction; wherein the preoperative part AR model of the patient is formed by performing three-dimensional reconstruction by using preoperative medical diagnosis image information of the patient through an operation planning workstation; the pre-operation part AR model is subjected to surgical operation simulation; the doctor AR operation terminal is further used for executing operation simulation operation on the pre-operation part AR model, and the executed process is displayed on the doctor AR display terminal from the view angle that the doctor watches the AR model; and the patient AR operation terminal is used for synchronously displaying the execution process on the patient AR display terminal from a visual angle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of doctor-patient communication processing, and in particular to a doctor-patient communication system. Background Art

[0002] Before performing some difficult surgeries, in order to improve patients' cognition, doctors will communicate with patients before the operation, explain the surgical methods and the benefits and corresponding risks of the surgery, and thus improve patients' compliance with the surgery. However, because doctors and patients have different levels of medical cognition, there are certain communication barriers. The existing solution is for doctors to display visual lesions and verbally describe the surgical process with the help of visual lesions, but this method is not intuitive enough, patients may find it difficult to understand, and the communication effect is average. Summary of the invention

[0003] Based on this, it is necessary to provide a doctor-patient communication system to address the above technical issues.

[0004] The doctor-patient communication system provided in this application includes: a doctor AR display terminal, a patient AR display terminal, a doctor AR operation terminal and a patient AR operation terminal;

[0005] The doctor AR operation terminal is used to obtain the patient's preoperative AR model from the surgery planning workstation in response to the doctor's instruction; wherein the patient's preoperative AR model is formed by the surgery planning workstation performing three-dimensional reconstruction using the patient's preoperative medical diagnostic imaging information; the preoperative AR model is subjected to a simulation of the surgical operation;

[0006] The doctor AR operation terminal is further used to perform a surgical simulation operation on the preoperative site AR model, and display the execution process on the doctor AR display terminal from the doctor's perspective of viewing the AR model;

[0007] The patient AR operation terminal is used to synchronously display the execution process on the patient AR display terminal at the viewing angle.

[0008] In one embodiment, the surgical simulation operation includes a surgical approach opening simulation operation;

[0009] The doctor AR operation terminal is also used for:

[0010] Determining a simulated surgical incision location on the AR model of the preoperative site;

[0011] The surgical incision simulation is hidden to perform a surgical approach opening simulation operation on the preoperative site AR model.

[0012] In one embodiment, when the surgical planning workstation forms the AR model of the patient's preoperative part, the steps specifically include:

[0013] Performing three-dimensional reconstruction using the preoperative medical diagnostic imaging information of the patient to obtain a three-dimensional model of the patient's preoperative anatomical structure;

[0014] Generating a tissue region that fits the exterior of the three-dimensional model of the patient's preoperative anatomical structure;

[0015] Based on the tissue region and the three-dimensional model of the patient's preoperative anatomical structure, obtaining a preoperative AR model of the site;

[0016] When the surgical planning workstation generates a surgical incision simulation on the AR model of the preoperative part, the steps specifically include:

[0017] Determine an area on the tissue area of ​​the AR model of the preoperative site that intersects with the geometric body;

[0018] The intersection area is used as the surgical incision simulation site.

[0019] In one embodiment, when the opening simulation operation of the surgical approach is a bone window opening simulation operation of a neurosurgery operation,

[0020] When the surgical planning workstation generates a tissue region that fits the outside of the three-dimensional model of the patient's preoperative anatomical structure, it specifically includes:

[0021] A bone growth tool is used to generate a fitted skull tissue region on the three-dimensional model of the patient's preoperative anatomical structure based on the patient's bone tissue imaging data.

[0022] In one embodiment, before determining the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body, the surgical planning workstation is further configured to:

[0023] Obtain the size of the surgical incision planned preoperatively;

[0024] A geometric body having a cross-section that matches the size of the surgical incision is determined.

[0025] In one embodiment, when the surgical planning workstation determines the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body, it specifically includes:

[0026] Moving the geometric body onto the tissue region of the AR model of the preoperative site;

[0027] When the geometric body and the tissue region intersect, a region on the tissue region of the preoperative site AR model that intersects with the geometric body is determined.

[0028] In one embodiment, the doctor AR operation terminal and / or the surgery planning workstation are further used to:

[0029] The surgical incision is simulated to be in a wound area adjusted to have suture marks after surgery;

[0030] Generate prompt information for the wound area; the prompt information includes precautions for the rehabilitation stage of the wound area;

[0031] The patient AR operation terminal is used to display prompt information for the wound area.

[0032] In one embodiment, the doctor AR operation terminal is further used to create a virtual conference room to synchronously transmit the executed process to the patient AR operation terminal that joins the virtual conference room;

[0033] The patient AR operation terminal is used to join the virtual conference room to synchronously display the execution process on the patient AR display terminal.

[0034] The virtual conference room is used to realize data synchronization and sharing between the doctor AR operation terminal and the patient AR operation terminal based on the data connection between the two. The data connection is established in one or more of the following ways: based on a local area network, based on various standard communication networks, and based on a direct connection through a data line.

[0035] In one embodiment, the patient AR operation terminal is used to request the doctor AR operation terminal for AR model editing permission, and if the request is allowed, edit the displayed preoperative site AR model in response to the patient's operation.

[0036] In one embodiment, the editing includes one or more of cutting, coloring, scaling, moving, deconstructing, hiding and marking.

[0037] The above-mentioned doctor-patient communication system uses AR (augmented reality) technology to intuitively display the surgical process to the patient, which is easy for the patient to understand; and reuses the three-dimensional reconstruction function of the existing surgical planning workstation to reduce the design complexity of the doctor-patient communication system. Specifically, the doctor's AR operation terminal is used to obtain the patient's preoperative site AR model from the surgical planning workstation in response to the doctor's instructions; wherein the patient's preoperative site AR model is formed by the surgical planning workstation using the patient's preoperative medical diagnostic imaging information to perform three-dimensional reconstruction; the preoperative site AR model is simulated by the surgical operation; the doctor's AR operation terminal is also used to perform surgical simulation operations on the preoperative site AR model, and display the execution process on the doctor's AR display terminal from the doctor's perspective of viewing the AR model; the patient's AR operation terminal is used to synchronously display the execution process on the patient's AR display terminal from the perspective. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the architecture of a doctor-patient communication system in one embodiment;

[0039] Figure 2 A schematic diagram of the interaction process of each device in an embodiment;

[0040] Figure 3 The workflow of a doctor-patient communication system in a preoperative communication scenario in one embodiment;

[0041] Figure 4 A schematic diagram of a surgical simulation process in one embodiment;

[0042] Figure 5 The figure is a schematic diagram of the application process of the postoperative rehabilitation suggestion function in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0045] This application provides a doctor-patient communication system, such as Figure 1 As shown, the doctor-patient communication system includes: a doctor AR display terminal, a patient AR display terminal, a doctor AR operation terminal and a patient AR operation terminal.

[0046] The doctor AR operation terminal is used to obtain the patient's preoperative part AR model from the surgery planning workstation in response to the doctor's instruction;

[0047] The AR model of the patient's preoperative part is formed by three-dimensional reconstruction by the surgical planning workstation using the patient's preoperative medical diagnostic imaging information; the AR model of the preoperative part is subjected to a simulation of the surgical operation.

[0048] The patient's preoperative medical diagnostic imaging information can be data in dicom format, including but not limited to: computed tomography (CT), magnetic resonance imaging (MRI), CT angiography (CTA), diffusion tensor imaging (DTI-MRI) and other multimodal imaging data of the patient's surgical site.

[0049] The doctor AR operation terminal is further used to perform a surgical simulation operation on the preoperative site AR model, and display the execution process on the doctor AR display terminal from the doctor's perspective of viewing the AR model;

[0050] The patient AR operation terminal is used to synchronously display the execution process on the patient AR display terminal at the viewing angle.

[0051] Among them, the doctor AR operation terminal and the patient AR operation terminal can be integrated on the same operation terminal, or they can be two independent operation terminals.

[0052] The above-mentioned doctor-patient communication system uses AR (augmented reality) technology to intuitively display the surgical process to patients, which is more intuitive and easy for patients to understand; and reuses the three-dimensional reconstruction function of the existing surgical planning workstation to reduce the design complexity of the doctor-patient communication system.

[0053] Furthermore, doctors and patients can watch the surgical simulation operation performed on the AR model of the preoperative site in the same virtual conference room. Specifically, the doctor's AR operation terminal is also used to create a virtual conference room to synchronously transmit the execution process to the patient's AR operation terminal that joins the virtual conference room; the patient's AR operation terminal is used to join the virtual conference room to synchronously display the execution process on the patient's AR display terminal.

[0054] The virtual conference room is used to realize data synchronization and sharing between the doctor AR operation terminal and the patient AR operation terminal based on the data connection between the two. The data connection is established in one or more of the following ways: based on a local area network, based on various standard communication networks, and based on a direct connection through a data line.

[0055] In a face-to-face medical scenario, the doctor and the patient are in the same room in actual physical space, such as a conference room in a hospital, and use one or more AR devices for doctor-patient communication. At this time, the virtual conference room is built on the local area network of the physical space; when the doctor and the patient are in the same conference room in the hospital, the local area network can be the hospital's intranet.

[0056] In a telemedicine scenario, the doctor and the patient are far apart in actual physical space. For example, the doctor is in Province A and the patient is in Province B. In this scenario, the doctor can remotely direct the patient's local doctor or surgical robot to perform surgery. The virtual conference room can be set up on (but not limited to) a 5G communication network.

[0057] Other unknown or mixed situations, such as direct connection via a data cable that is not connected to the Internet, or indirect data exchange between the doctor and the patient's AR devices through other proxy servers.

[0058] Reference Figure 2 and Figure 3 After the patient is imaged, preoperative medical imaging diagnostic information in DICOM format can be obtained, and the preoperative medical imaging diagnostic information is sent to the surgery planning workstation; the surgery planning workstation uses the preoperative medical imaging diagnostic information to perform three-dimensional reconstruction to obtain the preoperative AR model of the site; the doctor can use the surgery planning workstation to plan the surgery. After the surgery is planned, the surgical operation is simulated on the AR model of the preoperative site according to the surgical steps obtained in the plan. The simulation process can be recorded to form relevant information about the surgery simulation operation.

[0059] Before doctor-patient communication, under the doctor's operation, the doctor's AR operation terminal can obtain the preoperative AR model of the site and related information of the surgical simulation operation from the surgical planning workstation, and establish a virtual conference room, and the patient's AR operation terminal can join the virtual conference room.

[0060] The doctor controls the doctor's AR operation terminal to call out the AR model of the preoperative site in the virtual conference room; after calling out, doctor-patient communication begins. At this time, the doctor's AR operation terminal can perform surgical simulation operations on the AR model of the preoperative site based on relevant information of the surgical simulation operation, and display the execution process on the doctor's AR display terminal and simultaneously send it to the patient's AR operation terminal that has joined the virtual conference room.

[0061] The patient AR operation terminal displays the received execution process on the patient AR display terminal, so that the doctor and the patient can synchronously watch the execution process of the surgical simulation operation on the AR model of the preoperative site.

[0062] In addition, during the execution of the operation of displaying the surgical model, the doctor can explain the risks associated with the operation. If the patient does not understand something, the patient can also ask the doctor questions with the help of the patient's AR operation terminal. During the Q&A stage between the patient and the doctor, the patient can use the AR model editing function to assist in communication. In the case where the AR model editing function of the patient's AR operation terminal has not been released by the doctor's AR operation terminal, the patient's AR operation terminal can first request the doctor's AR operation terminal to release the AR model editing function, and the doctor's AR operation terminal responds to the request and releases the AR model editing function. When both parties agree on the results of the communication, the preoperative doctor-patient communication ends, and the operation can be performed after the end.

[0063] In one embodiment, the patient AR operation terminal is used to request the doctor AR operation terminal for AR model editing permission, and if the request is allowed, edit the displayed preoperative site AR model in response to the patient's operation.

[0064] Furthermore, editing includes one or more of cutting, coloring, scaling, moving, deconstructing, hiding and marking.

[0065] In one embodiment, the surgical simulation operation includes an opening simulation operation of a surgical approach; the doctor's AR operation terminal is also used to determine a surgical incision simulation location on the preoperative site AR model; and hide the surgical incision simulation location to perform the opening simulation operation of the surgical approach on the preoperative site AR model.

[0066] Furthermore, when the surgical planning workstation forms the AR model of the patient's preoperative part, it specifically includes: using the patient's preoperative medical diagnostic imaging information to perform three-dimensional reconstruction to obtain a three-dimensional model of the patient's preoperative anatomical structure; generating a tissue area that fits the outside of the patient's preoperative anatomical structure three-dimensional model; and obtaining a preoperative part AR model based on the tissue area and the patient's preoperative anatomical structure three-dimensional model.

[0067] When the surgical planning workstation generates a surgical incision simulation on the preoperative AR model, it specifically includes: determining an area on the tissue area of ​​the preoperative AR model that intersects with a geometric body; and using the intersecting area as the surgical incision simulation.

[0068] Doctors use the operation panel to select preoperative medical diagnostic imaging information under a single or multiple modalities under the patient's name for application loading. The surgical planning workstation can automatically complete multi-modal data fusion according to each modality, automatically identify and segment the patient's common tissues, and then reconstruct a complete 3D model of the patient's preoperative anatomical structure composed of multiple independent 3D structural blocks. The 3D structural blocks are classified and named according to the actual tissue anatomical structure, and doctors can select, stain, and rename any 3D structural block individually.

[0069] The three-dimensional structural blocks obtained by three-dimensional reconstruction are not editable, and doctors cannot perform segmentation operations on the three-dimensional structural blocks obtained by three-dimensional reconstruction. Based on this, the present application provides a method to generate a fitted tissue area on the three-dimensional structural block obtained by three-dimensional reconstruction. The tissue area can be edited, and doctors can perform segmentation operations on the tissue area. The subsequently generated tissue area can be called a three-dimensional structural block obtained by non-three-dimensional reconstruction; subsequently, a preoperative site AR model is obtained based on the tissue area and the patient's preoperative anatomical structure three-dimensional model. The preoperative site AR model includes not only the three-dimensional structural blocks obtained by three-dimensional reconstruction, but also the tissue area obtained by non-three-dimensional reconstruction. The editable minimum unit of the preoperative site AR model is the tissue area, and compared with the patient's preoperative anatomical structure three-dimensional model, the editable granularity is finer.

[0070] Next, the doctor uses the surgery planning workstation to segment the surgical incision simulation according to the above steps. The surgical incision simulation can be hidden separately, that is, when the surgical incision simulation is hidden, other parts of the tissue area still remain displayed.

[0071] Furthermore, in the case where the opening simulation operation of the surgical approach is a bone window simulation operation for neurosurgery, when the surgical planning workstation generates a tissue area that fits the outside of the patient's preoperative anatomical structure three-dimensional model, it specifically includes: using a bone growth tool to generate a fitting skull tissue area on the patient's preoperative anatomical structure three-dimensional model based on the patient's bone tissue imaging data.

[0072] Furthermore, before determining the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body, the surgical planning workstation is also used to: obtain the size of the surgical incision planned preoperatively; and determine a geometric body whose cross-section matches the size of the surgical incision.

[0073] Furthermore, when the surgical planning workstation determines the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body, it specifically includes: moving the geometric body to the tissue area of ​​the preoperative site AR model; and when the geometric body and the tissue area intersect, determining the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body.

[0074] Taking the bone window operation in neurosurgery as an example, combined with Figure 4 introduce:

[0075] First, generate the skull tissue region. The doctor can select the 3D model of the patient's preoperative anatomical structure obtained by 3D reconstruction in the surgical planning workstation, select the 3D structural block of the skull in the model, and then select the bone growth tool in the surgical planning workstation to automatically generate a fitting skull tissue region in the 3D structural block of the skull. The skull tissue region can be defined as the 3D structural block of the skull obtained by non-3D reconstruction.

[0076] Next, a three-dimensional structure block of "bone flap" is generated. The doctor can use the three-dimensional model editing tool of the surgical planning workstation to select a cylinder of appropriate size according to the actual size of the bone window planned before surgery, and move the selected cylinder to the skull tissue area and intersect it. The intersection area is the location of the bone window in the preoperative plan. The doctor then needs to select this intersection area, assign it a new three-dimensional structure block, and name it "bone flap". "Bone flap" is a special name for the surgical incision in the bone window operation in neurosurgery.

[0077] In addition, the surgical planning workstation can generate PLY files that can be read by AR devices. After the three-dimensional reconstruction of patient data and preoperative planning are completed, the doctor can use the surgical planning workstation to convert the patient data into a PLY data format that can be read and edited by the AR device, and send it to the associated AR device. AR devices include AR operation terminals and AR display terminals.

[0078] Use AR devices to assist doctors and patients in preoperative communication. The doctor first uses the doctor AR display terminal and / or the doctor operation terminal to establish a virtual conference room. The audience (including patients or other medical staff) can use another one or more AR display terminals to join this virtual conference room, share the AR model viewing perspective, and view the same preoperative site AR model.

[0079] During the doctor-patient communication, the doctor can hide the "bone flap" three-dimensional structure block through the doctor's AR operation terminal to show the patient the approach options for this operation. Through the hidden "bone flap" area, the patient can see other three-dimensional structures displayed inside the skull, forming the effect of a "bone window surgical field of view".

[0080] The 3D structural blocks of other tissue parts in the AR model of the preoperative part can also be freely hidden, displayed, stained and edited according to the demonstration requirements to achieve the effect of highlighting and focusing. In certain specific cases, the audience, including patients, can also request the doctor to release the AR model editing authority and use their own operating terminal to edit the AR model of the preoperative part that shares the user's perspective for mutual discussion.

[0081] After the preoperative doctor-patient communication is completed, the operation is performed. After the operation is completed, the patient will undergo another medical diagnostic image scan. The doctor-patient communication system provided by this application also provides an AR visualization function for postoperative rehabilitation information: generating postoperative medical imaging diagnostic information of the patient's surgical site, and the postoperative medical imaging diagnostic information can be in dicom format.

[0082] In one embodiment, the doctor's AR operation terminal and / or the surgical planning workstation are also used to: simulate the surgical incision to be adjusted to a wound area with suture marks after surgery; generate prompt information for the wound area; the prompt information includes precautions for the rehabilitation stage of the wound area; the patient's AR operation terminal is used to display prompt information for the wound area.

[0083] Figure 5 An example is given to show the application process of the postoperative AR rehabilitation suggestion function. Specifically, after the operation is completed, the doctor can mark the postoperative rehabilitation suggestions on the AR model of the patient's postoperative part. The patient can download or copy his or her own AR model file of the postoperative part (including the diagnosis and treatment information annotation). During the rehabilitation period, he or she can use his or her own AR operation terminal / AR display terminal (or borrow the hospital's) to review the surgical outcomes and related rehabilitation risks, and understand the postoperative rehabilitation suggestions.

[0084] The doctor imports the postoperative medical imaging diagnostic information into the surgical planning workstation again, repeats the above introduction, and generates a three-dimensional model of the patient's postoperative anatomical structure. In the preoperative planning, the site of the bone window opened during the operation will become a wound area after the operation, with suture marks (such as titanium nails, protrusions, depressions, etc.).

[0085] Doctors can follow the steps of generating a 3D structural block of a "bone flap" to generate a 3D structural block at the wound site of the 3D model of the patient's postoperative anatomical structure and name it "wound". Doctors can use the surgical planning workstation or the doctor's AR operation terminal to select the "wound" area and use the "AR sticky note" tool to generate a text record, which can record the precautions for the rehabilitation stage of this wound (including but not limited to medication, sleeping position, compression warnings, rehabilitation cycle, etc.).

[0086] Doctors can also allow the surgical planning workstation to access information about a single or multiple patients in the hospital information system (HIS). The surgical planning workstation automatically generates postoperative rehabilitation information for the corresponding patients based on the doctor's orders and integrates it into the AR model file of the corresponding patient to form an "AR sticky note."

[0087] Doctors can use the surgery planning workstation or the doctor's AR operation terminal to retrieve the patient's files, and repeatedly view the "AR sticky note" through the doctor's AR display terminal to assist in patient follow-up. Patients can also copy their own AR models after surgery, repeatedly study their postoperative anatomical changes and the doctor's advice information in the "AR sticky note", so as to familiarize themselves with and clarify the doctor's postoperative rehabilitation advice and deepen their understanding of their physical condition.

[0088] This article proposes an interactive doctor-patient communication system that allows doctors to show patients the customized surgical process, as well as possible benefits and risks based on their actual condition before surgery, thereby improving patient cooperation and surgical cognition and overall treatment efficiency. After surgery, patients can review rehabilitation suggestions in a concrete way and doctors can engage in academic exchanges.

[0089] The doctor-patient communication system proposed in this application is an interactive doctor-patient communication system based on AR technology, which allows doctors to show patients the entire surgical process, as well as possible benefits and risks based on their actual condition before surgery, thereby improving patient cooperation and surgical cognition and improving surgical efficiency. After surgery, patients can review rehabilitation suggestions in a concrete way, and doctors can have academic exchanges.

[0090] The doctor-patient communication system generates a virtual color interactive model based on the patient's actual professional medical diagnosis information. Patients can watch the doctor's operations on this virtual model, including but not limited to cutting, staining, scaling, moving, deconstructing, hiding, labeling, etc., to deepen their understanding of surgery-related information; on the other hand, patients can also perform corresponding operations on this model according to the doctor's requirements, and doctors can understand the patient's understanding of the surgery based on the patient's operations. After surgery, patients can review their AR models to gain an in-depth understanding of their condition and postoperative rehabilitation recommendations.

[0091] The doctor-patient communication system proposed in this application has the following advantages:

[0092] 1) The virtual, colorful, interactive AR model is based on the patient’s actual medical diagnosis information and can match the actual conditions of different patients, thus eliminating misunderstandings during communication;

[0093] 2) Unlike traditional planar anatomical image information, the virtual AR model constructed by the system can be freely moved, scaled, sliced, deconstructed, stained, partitioned, and annotated, making it easier for people without medical knowledge to understand anatomical information;

[0094] 3) The patient and the doctor each hold an AR terminal (or one AR and one other device, such as an iPad), and can perform editing operations such as moving, scaling, cutting, deconstructing, and staining partitions on the same virtual AR model. The doctor can judge the patient's understanding of the surgery by observing the patient's operations on the AR model, and can flexibly adjust the communication progress and method to achieve efficient communication;

[0095] 4) Pre-processing can more accurately simulate customized surgical procedures, making communication and presentation more accurate and efficient;

[0096] 5) Although the constructed virtual AR model provides the real size ratio of organs and surgery, its visual effect is more cartoon-like than the actual anatomical model, which reduces the fear of professional anatomical information to patients, thereby improving patients' understanding efficiency and cooperation with surgery;

[0097] 6) It can cover the entire perioperative doctor-patient communication scenario, not just the preoperative period. Postoperative imaging tests can update the patient's AR file, and the postoperative AR model will include the doctor's rehabilitation advice to the patient. This file can be downloaded or copied to the patient himself, helping the patient to deepen his understanding of postoperative risks and rehabilitation processes, and also facilitating the doctor's postoperative follow-up;

[0098] 7) Allow doctors to quickly create AR simulation demonstrations of surgical operations on the spot based on the different conditions of patients.

[0099] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0100] It should be noted that the patient information and data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the patients.

[0101] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0102] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A doctor-patient communication system, characterized in that: The doctor-patient communication system includes: a doctor AR display terminal, a patient AR display terminal, a doctor AR operation terminal and a patient AR operation terminal; The doctor AR operation terminal is used to obtain the patient's preoperative AR model from the surgery planning workstation in response to the doctor's instruction; wherein the patient's preoperative AR model is formed by the surgery planning workstation performing three-dimensional reconstruction using the patient's preoperative medical diagnostic imaging information; the preoperative AR model is subjected to a simulation of the surgical operation; The doctor AR operation terminal is further used to perform a surgical simulation operation on the preoperative site AR model, and display the execution process on the doctor AR display terminal from the doctor's perspective of viewing the AR model; The patient AR operation terminal is used to synchronously display the execution process on the patient AR display terminal at the viewing angle.

2. The doctor-patient communication system according to claim 1, characterized in that: The surgical simulation operation includes a surgical approach opening simulation operation; The doctor AR operation terminal is also used for: Determining a simulated surgical incision location on the AR model of the preoperative site; The surgical incision simulation is hidden to perform a surgical approach opening simulation operation on the preoperative site AR model.

3. The doctor-patient communication system according to claim 2, characterized in that: When the surgical planning workstation forms the AR model of the patient's preoperative part, it specifically includes: Performing three-dimensional reconstruction using the preoperative medical diagnostic imaging information of the patient to obtain a three-dimensional model of the patient's preoperative anatomical structure; Generating a tissue region that fits the exterior of the three-dimensional model of the patient's preoperative anatomical structure; Based on the tissue region and the three-dimensional model of the patient's preoperative anatomical structure, obtaining a preoperative AR model of the site; When the surgical planning workstation generates a surgical incision simulation on the AR model of the preoperative part, the steps specifically include: Determine an area on the tissue area of ​​the AR model of the preoperative site that intersects with the geometric body; The intersection area is used as the surgical incision simulation site.

4. The doctor-patient communication system according to claim 3, characterized in that: When the surgical approach opening simulation operation is a neurosurgery bone window simulation operation, When the surgical planning workstation generates a tissue region that fits the outside of the three-dimensional model of the patient's preoperative anatomical structure, it specifically includes: A bone growth tool is used to generate a fitted skull tissue region on the three-dimensional model of the patient's preoperative anatomical structure based on the patient's bone tissue imaging data.

5. The doctor-patient communication system according to claim 3, characterized in that: Before determining the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body, the surgical planning workstation is further used to: Obtain the size of the surgical incision planned preoperatively; A geometric body having a cross-section that matches the size of the surgical incision is determined.

6. The doctor-patient communication system according to claim 3, characterized in that: When the surgical planning workstation determines the area on the tissue area of ​​the preoperative site AR model that intersects with the geometric body, it specifically includes: Moving the geometric body onto the tissue region of the AR model of the preoperative site; When the geometric body and the tissue region intersect, a region on the tissue region of the preoperative site AR model that intersects with the geometric body is determined.

7. The doctor-patient communication system according to claim 2, characterized in that: The doctor's AR operation terminal and / or the surgery planning workstation are also used for: The surgical incision is simulated to be in a wound area adjusted to have suture marks after surgery; Generate prompt information for the wound area; the prompt information includes precautions for the rehabilitation stage of the wound area; The patient AR operation terminal is used to display prompt information for the wound area.

8. The doctor-patient communication system according to claim 1, characterized in that: The doctor AR operation terminal is further used to create a virtual conference room to synchronously transmit the executed process to the patient AR operation terminal that joins the virtual conference room; The patient AR operation terminal is used to join the virtual conference room to synchronously display the execution process on the patient AR display terminal.

9. The doctor-patient communication system according to claim 1, characterized in that: The patient AR operation terminal is used to request the doctor AR operation terminal for AR model editing permission, and edit the displayed preoperative part AR model in response to the patient's operation when the request is allowed.

10. The doctor-patient communication system according to claim 9, characterized in that: The editing includes one or more of cutting, coloring, scaling, moving, deconstructing, hiding and marking.