AR (Augmented Reality)-based doctor-seeing whole-process guiding system and method and medium

Abstract

The invention relates to the technical field of virtual simulation, in particular to an AR (Augmented Reality)-based medical-seeing whole-process guiding system and method and a medium, the system is accessed to a hospital management system and an electronic medical record system, and the system comprises a cloud server used for acquiring a current medical-seeing demand of a target user based on medical-seeing information of the electronic medical record system and sending the current medical-seeing demand to the target user; based on the treatment demand and the current positioning, generating a guide line, and transmitting the guide line to the terminal equipment; the AR equipment comprises a camera and is used for acquiring a field image and transmitting the field image to the terminal equipment; the terminal equipment is used for generating a virtual guidance picture based on the guidance line and the field image and sending the virtual guidance picture to the AR equipment; and the AR equipment is also used for presenting the virtual guiding picture, so that the target user sees a doctor according to the virtual guiding picture, the guiding picture is realized in a virtual enhancement mode, the patient can reach the doctor seeing position according to the guiding picture, the doctor seeing efficiency is improved, and the doctor seeing experience is also improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of virtual simulation, and in particular to an AR-based medical treatment whole-process guiding system, method and medium. BACKGROUND

[0002] Existing medical treatment guiding is mostly in the form of sign guiding, text guiding and the like. However, due to the complexity of hospital layout and the scattered distribution of various departments, these guiding modes cannot effectively guide patients to the destination. Therefore, these guiding modes lack flexibility.

[0003] Although artificial guiding can be used, it cannot meet the needs of patients at any time due to the limitation of manpower and time. For patients who are visiting a hospital for the first time or are not feeling well, it is particularly difficult to quickly and accurately find various medical treatment locations and understand the medical treatment process, which not only increases the medical treatment time and experience cost of patients, but also causes anxiety of patients and affects the medical treatment experience.

[0004] Therefore, how to guide patients to visit a hospital and improve the medical treatment experience of patients is a technical problem to be solved at present. SUMMARY

[0005] In view of the above problems, the present application provides an AR-based medical treatment whole-process guiding system, method and medium which overcome the above problems or at least partially solve the above problems.

[0006] In a first aspect, the present application provides an AR-based medical treatment whole-process guiding system connected with a hospital management system and an electronic medical record system, characterized in that it comprises:

[0007] a cloud server configured to obtain the current medical treatment demand of a target user based on the medical treatment information of the electronic medical record system, and generate a guiding route based on the medical treatment demand and the current positioning, and transmit the guiding route to a terminal device;

[0008] the AR device comprises a camera configured to obtain a live image and transmit the live image to the terminal device;

[0009] the terminal device is configured to generate a virtual guiding picture based on the guiding route and the live image, and send the virtual guiding picture to the AR device;

[0010] the AR device is further configured to present the virtual guiding picture so that the target user visits a hospital according to the virtual guiding picture.

[0011] Preferably, the terminal device is further configured to:

[0012] obtain the current positioning of the target user and transmit the current positioning to the cloud server.

[0013] Preferably, the cloud server is specifically configured to:

[0014] When the medical treatment demand is the first visit to a doctor, the flow of the corresponding outpatient department inside the hospital is obtained based on a hospital management system;

[0015] Based on the flow and the current location, the length of time that the target user needs to wait is determined.

[0016] Preferably, the cloud server is specifically configured to:

[0017] When the medical treatment demand is to have an examination, the examination queue of the corresponding examination item inside the hospital is obtained based on a hospital management system;

[0018] Based on the examination queue, the medical treatment demand and the current location, the order of the examination items is determined.

[0019] Based on the order of the examination items, a guide route is generated, the guide route including a route from the current location to a first examination item, a route from the first examination item to a second examination item, and a route to an Nth examination item, N being the total number of the examination items.

[0020] Preferably, the terminal device is specifically configured to: when the medical treatment demand is to handle hospitalization procedures or to handle discharge procedures, obtain the procedure information of hospitalization or discharge through the cloud server, generate a virtual list of the procedure information based on the procedure information, and send the virtual list to the AR device for presentation;

[0021] When the target user places the prepared procedure entity in front of the AR device, the procedure entity is collected by a camera of the AR device and transmitted to the terminal device;

[0022] The terminal device checks the corresponding content on the virtual list of the procedure information based on the procedure entity and presents the virtual list through the AR device.

[0023] Preferably, the terminal device is configured to generate a virtual arrow corresponding to the live image based on the live image and the guide route, superimpose the virtual arrow on the live image, and generate a virtual guide picture.

[0024] Preferably, the terminal device is further configured to:

[0025] The virtual guide picture is transmitted to the server for saving, so as to be restored later.

[0026] In a second aspect, the application further provides an AR-based medical treatment whole-process guiding method applied to an AR-based medical treatment whole-process guiding system, the AR-based medical treatment whole-process guiding system being connected to a hospital management system and an electronic medical record system, and the AR-based medical treatment whole-process guiding method comprising:

[0027] Based on the visit information of the electronic medical record system, the current visit demand of the target user is obtained;

[0028] Based on the visit demand and the current positioning, a guide route is generated;

[0029] Obtain the on-site image;

[0030] Based on the guide route and the on-site image, a virtual guide picture is generated;

[0031] The virtual guide picture is presented, so that the target user visits according to the virtual guide picture.

[0032] In a third aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the method in the second aspect.

[0033] The one or more technical solutions in the embodiments of the present application have at least the following technical effects or advantages:

[0034] The present application provides an AR-based medical treatment whole-process guiding system, which is connected with a hospital management system and an electronic medical record system, and includes: a cloud server, which is used to obtain the current visit demand of a target user based on the visit information of the electronic medical record system, and generate a guide route based on the visit demand and the current positioning, and transmit the guide route to a terminal device; an AR device, which includes a camera and is used to obtain an on-site image and transmit the on-site image to the terminal device; and the terminal device, which is used to generate a virtual guide picture based on the guide route and the on-site image, and send the virtual guide picture to the AR device; and the AR device is further used to present the virtual guide picture, so that the target user visits according to the virtual guide picture, and the guide picture is realized in a virtual augmented manner, so that the patient can reach the visit position according to the guide picture, the visit efficiency is improved, and the visit experience is also improved. BRIEF DESCRIPTION OF DRAWINGS

[0035] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. Moreover, the same reference numerals in different drawings represent the same or similar elements. In the drawings:

[0036] Figure 1 The structure schematic diagram of the AR-based medical treatment whole-process guiding system in the embodiments of the present application is shown;

[0037] Figure 2 The schematic diagram of the virtual list of formalities in the embodiments of the present application is shown;

[0038] Figure 3 The schematic diagram of the virtual guide picture in the embodiments of the present application is shown

[0039] Figure 4 A step flow diagram of the AR-based medical treatment whole-process guiding method in the embodiment of the present application is shown. DETAILED DESCRIPTION

[0040] Exemplary embodiments of the present application will be described in greater detail below with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thoroughly and completely understood, and so that the scope of the present application will be completely conveyed to those skilled in the art.

[0041] Embodiment one:

[0042] The embodiment of the present application provides an AR-based medical treatment whole-process guiding system, which comprises: Figure 1 As shown in the figure, the AR-based medical treatment whole-process guiding system comprises:

[0043] a cloud server 101, configured to acquire a current medical treatment demand of a target user based on medical treatment information of an electronic medical record system, and generate a guiding route based on the medical treatment demand and a current positioning, and transmit the guiding route to a terminal device 102;

[0044] an AR device 103, comprising a camera, configured to acquire a live image and transmit the live image to the terminal device;

[0045] the terminal device 102, configured to generate a virtual guiding picture based on the guiding route and the live image, and send the virtual guiding picture to the AR device 103;

[0046] the AR device 103 is further configured to present the virtual guiding picture, so that the target user can be guided to the medical treatment according to the virtual guiding picture.

[0047] In a specific implementation, the terminal device 102 needs to acquire the current positioning of the target user first, and transmit the current positioning to the cloud server 101.

[0048] Before that, the cloud server 101 acquires the current medical treatment demand of the target user through the medical treatment information of the electronic medical record system.

[0049] The medical treatment demand herein has multiple cases, and according to different demands of patients, the medical treatment demand can be a first medical treatment demand for asking a doctor, a medical treatment demand for performing various examination items after asking a doctor, or a medical treatment demand for hospitalization or discharge according to a diagnosis result.

[0050] The medical treatment demands will be described respectively as follows:

[0051] When the medical treatment demand is the first medical treatment demand for asking a doctor, the traffic condition of a corresponding outpatient department inside a hospital is acquired based on a hospital management system;

[0052] Based on the flow condition and the current position, the length of time that the target user needs to wait is determined.

[0053] For the target patient who visits a hospital for the first time, the visiting demand is obtained from the moment the target patient steps into the hospital, that is, the first visit. The flow condition of the corresponding outpatient department inside the hospital is obtained through the hospital management system, specifically, the flow condition of each visiting department. According to the flow condition, the length of time that the target patient needs to wait corresponding to the current registration condition is obtained. Although the target user has made an appointment for the visiting time, there is a certain difference between the visiting time and the appointment time because the visiting time changes in real time. By determining the length of time that the target user needs to wait, the length of time is transmitted to the AR device 103 through the terminal device 102 and is presented through the AR device 103, so that the target user can determine the length of time in real time. The target user can reasonably arrange the time.

[0054] Next, when the visiting demand is to do an examination, the examination queuing condition of the corresponding examination item inside the hospital is obtained based on the hospital management system.

[0055] Based on the examination queuing condition, the visiting demand, and the current positioning, the sorting of the examination item is determined.

[0056] Based on the sorting of the examination item, a guide route is generated, which includes a route from the current positioning to the first examination item, a route from the first examination item to the second examination item, and a route to the Nth examination item, where N is the total number of examination items.

[0057] After the inquiry link of the first visit, the doctor will issue the corresponding examination item for the target user to examine. The queuing order of each examination is different. Specifically, the queuing condition of each examination item can be determined according to the hospital management system. According to the principle that the examination item with a higher queuing order is arranged for examination first and the examination item with a lower queuing order is arranged for examination later, the sorting of the examination item is generated in combination with the visiting demand of the target user and the current positioning.

[0058] In order to improve the efficiency of the target user in the hospital, that is, the efficiency of doing various examination items. Based on the sorting of the examination item, each route from the current position to the first examination item, from the first examination item to the second examination item, and to the Nth examination item is generated and planned. In the planning process, congested road sections can be avoided and smooth road sections can be selected.

[0059] After the target user has done various examinations, the corresponding diagnosis result is obtained, and the AR device 103 is also provided with a camera for acquiring a live image and transmitting it to the terminal device 102. Therefore, if the examination result indicates that hospitalization or discharge is required, the terminal device 102 is configured to: when the medical treatment requirement is hospitalization or discharge, obtain the hospitalization or discharge procedure information through the cloud server 101, generate a procedure list based on the procedure information, and send it to the AR device 103 of the target user for presentation;

[0060] When the target user places the prepared procedure entity in front of the AR device 103, the camera of the AR device collects the procedure entity and transmits it to the terminal device 102;

[0061] The terminal device 102 checks the corresponding content on the procedure virtual list based on the procedure entity and presents it through the AR device.

[0062] Specifically, when the target user needs to handle admission or discharge, the target user first needs to prepare the procedure information, and then needs to deliver the procedure information to the corresponding department for storage.

[0063] When preparing the corresponding procedure, the terminal device 102 obtains the hospitalization or discharge procedure information through the cloud server 101, specifically, the cloud server 101 obtains it through the accessed hospital management system. Next, the terminal device 102 generates a procedure virtual list based on the procedure information, and then sends it to the AR device 103 for presentation, as shown in Figure 2 .

[0064] In order to enable the target patient to accurately prepare the procedure information, when the target user prepares each procedure entity, the target user places it in front of the AR device 103 to be collected by the camera and transmitted to the terminal device 102. The terminal device 102 judges the correctness of the procedure entity, and when the correctness is determined, checks the corresponding content on the procedure virtual list. The target user can continue to prepare the next procedure, and the verification of each procedure is sent to the AR device 103 for presentation. Finally, the target patient is guided to the destination to deliver the procedure information, so as to quickly realize admission or discharge.

[0065] Next, regarding the presentation of the virtual guide picture, it is realized through the processing of the terminal device 101. The terminal device 102 is configured to:

[0066] Based on the live image and the guide route, a virtual arrow corresponding to the live image is generated, and the virtual arrow is superimposed on the live image to generate a virtual guide picture.

[0067] As shown in Figure 3As shown, according to the guide line and the current position of the target user, it is determined that the target user needs to go straight along the front road at this time, and a virtual arrow is displayed on the top of the front road in the live image, and a virtual guide picture is obtained by superimposing the picture. The virtual guide picture can be applied to various AR presentation modes of the guide line.

[0068] In order to trace the line condition later, the terminal device 102 is further configured to transmit the virtual guide picture to the cloud server 101 for storage, so as to restore later.

[0069] In an optional embodiment, the AR device further comprises a voice prompting module configured to provide voice guidance when the target user walks in the wrong direction.

[0070] In addition, a voice collection module is further included, configured to collect the question information of the target user, such as "how should I go" and "where should I go next", and provide corresponding voice answers through the voice prompting module.

[0071] Therefore, the system covers the whole process guidance of hospitalization or discharge before, during and after diagnosis, and provides personalized guidance services according to the real-time position, medical progress and personal information of the target user, such as queuing before diagnosis, line planning and examination instructions during diagnosis, and hospitalization procedures or discharge procedures after diagnosis, etc., which comprehensively provides guidance for the target user and improves the medical efficiency and experience.

[0072] The one or more technical solutions in the embodiment of the application have at least the following technical effects or advantages:

[0073] The application provides an AR-based medical whole-process guidance system connected with a hospital management system and an electronic medical record system, comprising: a cloud server configured to obtain the current medical needs of a target user based on the medical information of the electronic medical record system, generate a guide line based on the medical needs and the current positioning, and transmit the guide line to a terminal device; an AR device comprising a camera configured to obtain a live image and transmit the live image to the terminal device; the terminal device is configured to generate a virtual guide picture based on the guide line and the live image, and send the virtual guide picture to the AR device; the AR device is further configured to present the virtual guide picture, so that the target user can follow the virtual guide picture for medical treatment, and the virtual augmented guide picture enables the patient to reach the medical treatment position, improves the medical efficiency and experience.

[0074] Embodiment two

[0075] Based on the same inventive concept, the application further provides an AR-based medical whole-process guidance method applied to an AR-based medical whole-process guidance system connected with a hospital management system and an electronic medical record system, such as the AR-based medical whole-process guidance system shown in the above embodiment.Figure 4 As shown, comprising:

[0076] S401, based on the visit information of the electronic medical record system, obtaining the current visit demand of the target user;

[0077] S402, generating a guide route based on the visit demand and the current location;

[0078] S403, obtaining a live image;

[0079] S404, generating a virtual guide picture based on the guide route and the live image;

[0080] S405, presenting the virtual guide picture so that the target user visits according to the virtual guide picture.

[0081] In an optional implementation, further comprising:

[0082] When the visit demand is the first time to see a doctor, based on the hospital management system, obtaining the flow situation of the corresponding outpatient department inside the hospital;

[0083] Based on the flow situation and the current location, determining the length of time the target user needs to wait.

[0084] In an optional implementation, S402, comprising:

[0085] When the visit demand is to do an examination, based on the hospital management system, obtaining the examination queue situation of the corresponding examination item inside the hospital;

[0086] Based on the examination queue situation, the visit demand and the current location, determining the order of the examination items;

[0087] Based on the order of the examination items, generating a guide route, the guide route including a route from the current location to a first examination item, a route from the first examination item to a second examination item, and so on until a route to an Nth examination item, N being the total number of examination items.

[0088] In an optional implementation, further comprising:

[0089] When the visit demand is to handle inpatient procedures or to handle discharge procedures, obtaining inpatient or discharge procedure information, and based on the procedure information, generating a virtual list of procedure information;

[0090] When the target user places the prepared procedure information entity in front of the AR device, the camera of the AR device captures the procedure information entity;

[0091] Based on the procedure information entity, checking the corresponding content on the virtual list of procedure information, and presenting through the AR device.

[0092] In an optional implementation, S404 is specifically configured to:

[0093] generating a virtual arrow corresponding to the scene image based on the scene image and the guidance route;

[0094] The virtual arrow is superimposed on the on-site image to generate a virtual guidance picture.

[0095] In an optional implementation manner, after S404, the method further includes:

[0096] The virtual guidance screen is saved for later restoration.

[0097] Example 3

[0098] Based on the same inventive concept, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the above-mentioned AR-based full-process medical guidance method are implemented.

[0099] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general-purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing this type of system. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0100] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0101] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting an intention that the claimed invention requires more features than those explicitly recited in each embodiment. Rather, as reflected in each embodiment, inventive aspects lie in fewer than all the features of the individual embodiments previously disclosed. Accordingly, the claims that follow the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0102] Those skilled in the art will appreciate that the modules in the apparatuses in the embodiments can be adapted and placed in one or more apparatuses other than the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and furthermore can be split into multiple sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or process or device of any combination of the features disclosed in the specification (including the accompanying claims, abstract and drawings) can be taken, except that at least some of such features and / or processes or units are mutually exclusive, unless explicitly stated otherwise. Each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by alternative features providing the same, equivalent, or similar functions unless stated explicitly otherwise.

[0103] Furthermore, those skilled in the art will appreciate that different embodiments of the application can be implemented in a computer system for implementing the AR-based medical visit full-process guidance system according to the embodiments of the application. The computer system can include one or more processors, memory, and input / output interfaces. The memory can store computer program instructions for implementing the AR-based medical visit full-process guidance system according to the embodiments of the application. The input / output interfaces can be configured to receive input data and output data. The computer system can be configured to perform the methods described herein.

[0104] The various component embodiments of the application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. Those skilled in the art will appreciate that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components in the AR-based medical visit full-process guidance system according to the embodiments of the application. The application can also be implemented as a program (for example, a computer program and a computer program product) for performing part or all of the methods described herein. Such a program implementing the application can be stored on a computer readable medium, or can have one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0105] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices or means can be listed, comprising means which can be implemented by one and the same hardware item. The use of the word "a" or "an" does not exclude the presence of a plurality of such elements, nor does it imply that a single element is to be used.

Claims

1. An AR-based full-process medical guidance system, connected to the hospital management system and electronic medical record system, characterized by: include: The cloud server is used to obtain the current medical needs of the target user based on the medical information of the electronic medical record system, and generate a guidance route based on the medical needs and current location, and transmit it to the terminal device; The AR device includes a camera for acquiring on-site images and transmitting them to the terminal device; A terminal device, configured to generate a virtual guidance image based on the guidance route and the on-site image, and send the image to the AR device; The AR device is also used to present the virtual guidance screen so that the target user can see a doctor according to the virtual guidance screen.

2. The system according to claim 1, wherein The terminal device is further used for: Get the current location of the target user and transmit it to the cloud server.

3. The system according to claim 1, wherein: The cloud server is specifically used for: When the medical consultation demand is the first consultation with a doctor, the flow of the corresponding outpatient clinic in the hospital is obtained based on the hospital management system; Based on the traffic situation and current location, determine how long the target user needs to wait.

4. The system according to claim 1, wherein: The cloud server is specifically used for: When the medical need is to undergo an examination, based on the hospital management system, the examination queue status of the corresponding examination items within the hospital is obtained; Determine the order of examination items based on the examination queue status, the medical needs and the current location; Based on the order of the inspection items, a guidance route is generated, which includes a route from the current location to the first inspection item, a route from the first inspection item to the second inspection item, and finally a route to the Nth inspection item, where N is the total number of inspection items.

5. The system according to claim 1, wherein: The terminal device is specifically configured to: when the medical treatment requirement is hospitalization or discharge procedures, obtain hospitalization or discharge procedure information from a cloud server, generate a virtual list of procedure information based on the procedure information, and send the virtual list to the AR device for presentation; When the target user places the prepared procedure document entity in front of the AR device, the procedure document entity is captured by the camera of the AR device and transmitted to the terminal device; The terminal device checks corresponding content on the virtual list of procedure documents based on the procedure document entity and presents it through the AR device.

6. The system according to claim 1, wherein: The terminal device is used to generate a virtual arrow corresponding to the on-site image based on the on-site image and the guidance route, and superimpose the virtual arrow on the on-site image to generate a virtual guidance picture.

7. The system according to claim 6, wherein: The terminal device is further used for: The virtual guidance screen is transmitted to the server for storage so as to be restored later.

8. An AR-based full-process medical guidance method, applied to an AR-based full-process medical guidance system, wherein the AR-based full-process medical guidance system is connected to a hospital management system and an electronic medical record system, and is characterized in that: include: Obtain the target user's current medical needs based on the medical information in the electronic medical record system; Generate a guidance route based on the medical needs and current location; Acquire on-site images; generating a virtual guidance picture based on the guidance route and the on-site image; The virtual guidance screen is presented so that the target user can see a doctor according to the virtual guidance screen.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to claim 8 is implemented.

Images