Eye postoperative care simulation operation system based on mixed reality

Abstract

The invention discloses an eye postoperative nursing simulation operation system based on mixed reality, and relates to the technical field of postoperative nursing simulation, and the system comprises a simulation nursing platform module which is used for building a virtual nursing environment through an MR system; the operation simulation module is used for simulating the touch sense of real nursing operation through a force feedback handle and a touch sense feedback glove; and the motion capture module is used for capturing the motion trail of the user in the postoperative nursing operation and processing and analyzing the data. A virtual nursing environment is established through the MR technology so that a user can be trained in the environment close to the reality, the force feedback handle and the tactile feedback glove are adopted so that the user can obtain the real operation tactile sense, the motion capture module captures the operation track of the user to improve the operation skill, and by collecting and processing images in real time, the real-time training of the user can be achieved. And superposing the virtual nursing guidance information into an actual operation environment, and providing real-time guidance and feedback for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of postoperative care simulation, and in particular to an eye postoperative care simulation operating system based on mixed reality. Background Art

[0002] In recent years, mixed reality technology has developed rapidly and achieved remarkable results in many fields. Mixed reality technology combines augmented reality and virtual reality, captures real-world images through optical lenses, processes and enhances images through processors, and then presents the optimized images in an almost delay-free manner. This technology has gradually gained attention in the field of healthcare, especially in postoperative care of ophthalmic surgery. Mixed reality technology provides users and their families with a safe and intuitive simulation environment to better master nursing skills and reduce the incidence of postoperative complications.

[0003] Traditional postoperative eye care mainly relies on the guidance of medical staff and paper or video teaching materials. However, these methods have significant shortcomings. Although video teaching can intuitively demonstrate the care steps, it lacks interactivity. It is difficult for users and their families to improve their operational proficiency through practice. Medical staff often face heavy work pressure in clinical practice and it is difficult to provide sufficient time for detailed postoperative care guidance. Due to the lack of professional knowledge of users and their families, they may encounter problems such as eyelashes scratching the eyeball, incomplete eye ointment closure, and blood clots at the sutures in actual care. This not only affects postoperative recovery, but may also cause serious complications such as exposure keratitis. Summary of the invention

[0004] In view of the above-mentioned problems existing in the existing mixed reality-based postoperative eye care simulation operating system, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that the existing video teaching and paper guidance materials lack interactivity, and it is difficult for users and their families to effectively master nursing skills through these methods. Medical staff resources are tight and it is impossible to provide personalized and detailed nursing guidance for each user and family member. Users and their families are prone to operational errors during the nursing process due to lack of practical experience, which increases the risk of postoperative complications.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a mixed reality-based eye postoperative care simulation operating system, which comprises:

[0007] A simulation nursing platform module is used to establish a virtual nursing environment through the MR system;

[0008] An operation simulation module, which is used to simulate the touch of real nursing operations through a force feedback handle and a tactile feedback glove;

[0009] Motion capture module, used to capture the user's motion trajectory during postoperative care operations and process and analyze the data;

[0010] The MR display terminal module is used to superimpose virtual postoperative care guidance information into the actual operating environment and provide instant operation guidance and feedback.

[0011] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system described in the present invention, the MR system refers to a mixed reality system that integrates the real world and the virtual world by integrating hardware and software, and the MR system includes MR glasses, tactile sensors, cameras, software platforms, laser pulse transmitters and detectors;

[0012] The tactile sensor is used to collect tactile feedback data, and the camera is used to capture and pre-process the environment image in real time.

[0013] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system described in the present invention, wherein: the establishment of a virtual care environment includes designing virtual content according to the postoperative care process, and writing a virtual content design document, including specific steps, precautions and virtual interaction requirements for each care task, and the design document includes the visual effects, animation process and interaction method of the virtual content;

[0014] Use 3D modeling software to create 3D models related to postoperative care, including eye structures, care tools and environments, and create corresponding animations for each care task to show the correct operation steps and gestures;

[0015] Optimize the details and texture of the 3D model, adjust the smoothness and realism of the animation, integrate the designed virtual content into the MR system software platform, collect user experience and feedback, and optimize and adjust the 3D modeling based on the feedback;

[0016] Users select postoperative care scenarios through the virtual interface in the MR glasses, including eye sealing care, wound cleaning, eye drops and functional exercises. By selecting the type of postoperative care scenario, the corresponding virtual care scene is loaded, including a nursing bed, facial model and care tools.

[0017] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system described in the present invention, wherein: the tactile sense of simulating real care operations through the force feedback handle and the tactile feedback gloves refers to using the force feedback handle and the tactile feedback gloves to provide tactile feedback;

[0018] Select a force feedback handle with six degrees of freedom, including three-axis translation and three-axis rotation, and select a high-precision brushless DC motor as the driving source of the feedback handle, and transmit the force feedback data to the central processor through a high-speed serial bus;

[0019] Multi-point tactile feedback gloves are selected, and pressure point actuators are used as tactile feedback units. A tactile feedback unit is equipped on each finger and palm area. The tactile feedback unit simulates touch through vibration and pressure feedback. A distributed control system is used to independently control each tactile feedback unit. The distributed control system adjusts the tactile feedback in real time according to the user's operation, and transmits the tactile feedback data to the central processing unit through the wireless communication module.

[0020] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system of the present invention, wherein: the capturing of the user's motion trajectory in the postoperative care operation refers to using laser pulse emission and detection technology to capture the user's motion trajectory in the force feedback handle and tactile feedback gloves;

[0021] Install a laser pulse transmitter to cover the entire operation area with the emitted laser, and install a laser pulse detector on the force feedback handle and the tactile feedback glove;

[0022] Locate the laser pulse signal, set the laser pulse transmitter emission frequency to 30Hz, 40Hz and 50Hz respectively, and record the time t when each transmitter emits the pulse signal i , use the laser pulse detector to receive the laser pulse signal from the transmitter and record the receiving time t r , according to the emission and reception time, the laser pulse signal propagation time T is calculated, the formula is:

[0023] T=t r -t i

[0024] Where T represents the propagation time of the laser pulse signal, t r Represents the laser pulse signal receiving time, t i Represents the time when each transmitter emits a laser pulse signal;

[0025] Using the speed of light c, calculate the distance d from the detector to each emitter i , the formula is:

[0026] d i =c·T

[0027] Among them, d i represents the distance from the detector to each emitter, c represents the speed of light, and T represents the propagation time of the laser pulse signal;

[0028] The detector position data is obtained according to the calculated distance data;

[0029] Use laser pulse emitters and detectors to capture the movement trajectory of the user's hand and guide the user to perform hand movements through the system interface.

[0030] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system described in the present invention, the data processing and analysis refers to transmitting the calculated detector position data to the central processing unit, and after the central processing unit receives the data, it performs real-time analysis and processing of the motion trajectory to generate real-time feedback and guidance information of the user operation;

[0031] The analyzed and processed results are displayed to the user in real time through MR glasses, providing operation guidance and correction information. The system continuously tracks user operations and updates location data in real time.

[0032] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system described in the present invention, the method of using a camera to capture environmental images in real time and preprocessing the images includes:

[0033] Users practice eye care operations in a virtual care environment, using a high-resolution camera to capture images of the user's surroundings in real time;

[0034] The collected environmental images are preprocessed, including denoising, contrast enhancement and color correction. The preprocessed environmental images are transmitted to the MR glasses for image enhancement. The image enhancement algorithm is applied. The formula is:

[0035] E=β·R+γ

[0036] Wherein, E represents the enhanced image, R represents the original image, β represents the contrast adjustment coefficient, and γ represents the brightness adjustment value.

[0037] As a preferred solution of the mixed reality-based eye postoperative care simulation operating system described in the present invention, the virtual postoperative care guidance information is superimposed on the actual operating environment and the instant operation guidance and feedback is provided, including:

[0038] According to the nursing process, the corresponding virtual nursing guidance information is generated, and the transparency coefficient is used to set the transparency of the virtual nursing guidance information. The formula is:

[0039] I MR =I Real +α·I Virtual

[0040] Among them, I MR Represents a mixed reality image, I Real represents a real-world image, I Virtual represents a virtual image, α is the transparency coefficient;

[0041] Virtual care guidance information is superimposed on the enhanced environmental image for fusion, and objects in the virtual content are aligned with objects in the actual environment using depth information and spatial positioning technology.

[0042] A computer device comprises: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a mixed reality-based postoperative eye care simulation operating system when executing the computer program.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a mixed reality-based postoperative eye care simulation operating system.

[0044] The beneficial effects of the present invention are as follows: the present invention utilizes MR technology to establish a virtual nursing environment so that users can train in an environment close to the real one, adopts force feedback handles and tactile feedback gloves so that users can obtain a real operating touch, and the motion capture module captures the user's operating trajectory to improve operating skills. By real-time acquisition and processing of images, the virtual nursing guidance information is superimposed on the actual operating environment to provide users with instant guidance and feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0046] Figure 1 This is a structural diagram of a mixed reality-based postoperative eye care simulation operating system.

[0047] Figure 2 This is a flow chart of a mixed reality-based postoperative eye care simulation operating system. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1

[0052] Reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, and this embodiment provides an eye postoperative care simulation operating system based on mixed reality, and the eye postoperative care simulation operating system based on mixed reality includes:

[0053] S1, simulation nursing platform module, used to establish a virtual nursing environment through the MR system;

[0054] Specifically, the MR system refers to a mixed reality system that integrates the real world and the virtual world by integrating hardware and software, and the MR system includes MR glasses, tactile sensors, cameras, software platforms, laser pulse transmitters and detectors;

[0055] The tactile sensor is used to collect tactile feedback data, and the camera is used to capture and pre-process the environment image in real time.

[0056] The MR system can provide an immersive and interactive experience. By integrating the real world and the virtual world, it enhances the user's perception and operation capabilities of information. By integrating high-performance hardware and advanced software, it can provide users with a new, immersive mixed reality experience.

[0057] Going a step further, establishing a virtual care environment includes,

[0058] Design virtual content according to the postoperative care process and write virtual content design documents, including the specific steps, precautions and virtual interaction requirements for each nursing task. The design documents include the visual effects, animation process and interaction methods of the virtual content;

[0059] Wash your hands: Turn on the tap, wet your hands with warm water, take hand sanitizer, rub your hands for at least 20 seconds, covering all parts, rinse your hands thoroughly with clean water, dry your hands with a clean towel, and use a paper towel when turning off the tap;

[0060] Eye sealing care steps: the user wears MR glasses, and virtual hand washing steps are presented in the user's field of view. After washing hands, the user selects eye ointment through gestures and voice commands. The MR system simulates checking the expiration date and packaging integrity of the eye ointment. When the user is operating, the MR system displays the distance between the eye ointment tube and the eyeball in real time, and prompts the user to adjust to the optimal distance through color and sound. The MR system displays a virtual eyeball model in the user's field of view and demonstrates pulling down the lower eyelid and applying eye ointment. The user follows the virtual guidance and simulates applying eye ointment. The MR system monitors the user's gestures and operations in real time, collects feedback and adjustment suggestions, and displays virtual models of sterile gauze and 3M transparent tape in the user's field of view. The MR system demonstrates the process of covering the eyes and fixing the gauze and tape, and adjusts the tightness of the tape in real time.

[0061] Steps for applying eye ointment: select eye ointment according to treatment needs, open the package of sterile cotton swabs, take out the cotton swabs and place them on a clean sterile operating table for use, use the sterile cotton swabs to open the eyelashes, check whether there is dirt around the eyelashes, if there is dirt, use the sterile cotton swabs to gently wipe it clean, gently pull the user's lower eyelid with your hand to expose the conjunctival sac, use the other hand to squeeze out appropriate eye ointment and evenly apply it on the lower eyelid, relax the eyelids, gently close the eyes, take a piece of sterile gauze, cover the eyes, and use medical tape to stick the gauze around the eyelids to fix it;

[0062] For the wound cleaning step, the MR system automatically calculates the amount of disinfectant according to the user's requirements, monitors the user's operating force and time through sensors, and gives real-time feedback and adjustment suggestions. The user washes his hands according to the instructions in the virtual environment, prepares a bottle of 0.5% iodine tincture disinfectant solution and a sterile cotton swab, dips the sterile cotton swab in the iodine tincture solution, and gently wipes the wound from the inside to the outside, prepares a bottle of 0.9% saline, uses a dropper, and slowly drips the saline from the inner corner of the eye to the outer corner of the eye, cleans the conjunctiva, and checks the wound. If there is any foreign matter, carefully remove it with sterile tweezers, and gently dry the eye;

[0063] Eye drops: The user washes his hands according to the instructions in the virtual environment. The MR system prompts the user to select sodium hyaluronate eye drops and shows the appearance and correct use of sodium hyaluronate eye drops. The MR system prompts the user to place the nozzle of the eye drop bottle 2 cm away from the eyeball. In the virtual environment, the user can move the eye drop bottle to the specified distance by hand operation and see the distance scale in real time in the MR glasses. At the prompt of the MR system, the user gently pulls down the lower eyelid with one hand and holds the eye drop bottle 2 cm above the other hand. The user gently squeezes the bottle and drips the eye drops into the small bag formed by the lower eyelid. The MR system prompts the user to gently close the eyes and turn the eyeballs in all directions, tracking the eye movements in real time and giving visual feedback.

[0064] Functional training steps: the user wears MR glasses and tries to focus on the virtual sphere by moving the head and eyes. The MR system monitors the user's focus in real time and displays a moving path of the sphere in the virtual environment, including up, down, left, right and middle positions, and displays clear signs to guide the user's eye movement direction;

[0065] Use 3D modeling software to create 3D models related to postoperative care, including eye structures, care tools and environments, and create corresponding animations for each care task to show the correct operation steps and gestures;

[0066] Optimize the details and textures of the 3D model, adjust the smoothness and realism of the animation, integrate the designed virtual content into the MR system, collect user experience and feedback, and optimize and adjust the 3D modeling based on the feedback;

[0067] Users select postoperative care scenarios through the virtual interface in the MR glasses, including eye sealing care, wound cleaning, eye drops and functional exercises. By selecting the type of postoperative care scenario, the corresponding virtual care scene is loaded, including a nursing bed, user face model and care tools.

[0068] In a virtual environment, using 3D models and animation processes, users can practice repeatedly in a virtual environment to better master nursing skills. Through the MR system, users can clearly see each step and correct gestures, and can immerse themselves in the virtual environment of postoperative care, which helps to improve the accuracy of operations and deepen users' understanding and memory of the nursing process. By collecting user experience and feedback, 3D models and animations are adjusted and optimized in a timely manner to ensure the quality and applicability of teaching content. The development and use of this virtual nursing content will promote technological innovation in the field of medical care and help promote and apply advanced technologies in the medical industry.

[0069] S2, operation simulation module, used to simulate the touch of real nursing operations through force feedback handles and tactile feedback gloves;

[0070] Specifically, the tactile feedback of simulating real nursing operations through the force feedback handle and the tactile feedback gloves provides tactile feedback by using the force feedback handle and the tactile feedback gloves;

[0071] Select a force feedback handle with six degrees of freedom, including three-axis translation and three-axis rotation, and select a high-precision brushless DC motor as the driving source of the feedback handle, and transmit the force feedback data to the central processor through a high-speed serial bus;

[0072] Multi-point tactile feedback gloves are selected, and pressure point actuators are used as tactile feedback units. A tactile feedback unit is equipped on each finger and palm area. The tactile feedback unit simulates touch through vibration and pressure feedback. A distributed control system is used to independently control each tactile feedback unit. The distributed control system adjusts the tactile feedback in real time according to the user's operation, and transmits the tactile feedback data to the central processing unit through the wireless communication module.

[0073] The force feedback handle and tactile feedback gloves can simulate the force and touch in real nursing operations, allowing users to adjust their operations in time during training. This helps users master the correct force and technique during the operation and improve the accuracy of nursing operations. By recording the user's operation data during training, it provides data support for teaching and research, which helps to improve the training effect of nursing operation skills.

[0074] S3, motion capture module, used to capture the user's motion trajectory during postoperative care operations and process and analyze the data;

[0075] Specifically, capturing the user's motion trajectory during postoperative care operations refers to using laser pulse emission and detection technology to capture the user's motion trajectory in a force feedback handle and a tactile feedback glove;

[0076] Install a laser pulse transmitter to cover the entire operation area with the emitted laser, and install a laser pulse detector on the force feedback handle and the tactile feedback glove;

[0077] Locate the laser pulse signal, set the laser pulse transmitter emission frequency to 30Hz, 40Hz and 50Hz respectively, and record the time t when each transmitter emits the pulse signal i , use the laser pulse detector to receive the laser pulse signal from the transmitter and record the receiving time t r , according to the emission and reception time, the laser pulse signal propagation time T is calculated, the formula is:

[0078] T=t r -t i

[0079] Where T represents the propagation time of the laser pulse signal, t r Represents the laser pulse signal receiving time, t i Represents the time when each transmitter emits a laser pulse signal;

[0080] Using the speed of light c, calculate the distance d from the detector to each emitter i , the formula is:

[0081] d i =c·T

[0082] Among them, di represents the distance from the detector to each emitter, c represents the speed of light, and T represents the propagation time of the laser pulse signal;

[0083] The detector position data is obtained according to the calculated distance data;

[0084] Use laser pulse emitters and detectors to capture the movement trajectory of the user's hand and guide the user to perform hand movements through the system interface.

[0085] Laser pulse technology can provide high-precision positioning information to ensure the accuracy of the user's hand movement trajectory. By capturing the user's hand movement trajectory in real time, the MR system can provide immediate feedback to help users understand whether their operations are accurate and make timely adjustments, thereby improving training results. By accurately capturing the hand movement trajectory, the system can provide users with a more realistic and intuitive training experience, thereby improving the quality of training. The system can record the user's movement trajectory data, which can be used for subsequent analysis and evaluation. Laser pulse technology can adapt to different operating environments and operating areas, and has strong flexibility and adaptability.

[0086] Furthermore, processing and analyzing the data refers to transmitting the calculated detector position data to the central processing unit, which, after receiving the data, performs real-time analysis and processing of the motion trajectory to generate real-time feedback and guidance information for user operations;

[0087] The analyzed and processed results are displayed to the user in real time through MR glasses, providing operation guidance and correction information. The system continuously tracks user operations and updates location data in real time.

[0088] By displaying the results of analysis and processing to the user in real time, the system can provide immediate operational guidance and correction information to help users understand whether their operations are accurate in a timely manner during training and make corresponding adjustments. Real-time data processing and analysis can provide a more realistic and dynamic training environment, making training closer to actual work scenarios, thereby enhancing training effects. The MR system can adjust feedback information in real time according to the user's operating conditions and provide personalized training plans. At the same time, the system can also automatically adjust the training difficulty according to the user's progress to achieve adaptive training.

[0089] Furthermore, the use of a camera to capture an environmental image in real time and preprocessing the image includes:

[0090] Users practice eye care operations in a virtual care environment, using a high-resolution camera to capture images of the user's surroundings in real time;

[0091] The collected environmental images are preprocessed, including denoising, contrast enhancement and color correction. The preprocessed environmental images are transmitted to the MR glasses for image enhancement. The image enhancement algorithm is applied. The formula is:

[0092] E=β·R+γ

[0093] Wherein, E represents the enhanced image, R represents the original image, β represents the contrast adjustment coefficient, and γ represents the brightness adjustment value.

[0094] The process of capturing and preprocessing the user's surrounding environment in real time through a high-resolution camera, and then using MR glasses to enhance the image, has brought significant benefits to virtual nursing operation training. It not only improves the realism of the operation and the clarity of the visual feedback, enhances the training effect, but also ensures the safety of the training. The virtual training method combined with real-time image acquisition and processing provides users with an efficient, safe and interactive training solution.

[0095] S4, MR display terminal module, used to superimpose virtual postoperative care guidance information into the actual operation environment and provide instant operation guidance and feedback;

[0096] Specifically, virtual nursing guidance information is superimposed on the actual operation environment and provides immediate operation guidance and feedback, including:

[0097] According to the nursing process, the corresponding virtual nursing guidance information is generated, and the transparency coefficient is used to set the transparency of the virtual nursing guidance information. The formula is:

[0098] I MR =I Real +α·I Virtual

[0099] Among them, I MR Represents a mixed reality image, I Real represents a real-world image, I Virtual represents a virtual image, α is the transparency coefficient;

[0100] The virtual nursing guidance information is superimposed on the real-time collected environmental image for fusion, and the depth information is used to determine the position of the virtual content in the three-dimensional space and its relative position relationship with real-world objects. The spatial positioning technology is used to enable the MR system to track the position changes of the user and the device in real time and adjust the position and viewing angle of the virtual content, and align the objects in the virtual content with the objects in the actual environment.

[0101] Virtual nursing guidance information is superimposed on the actual operating environment and integrated through transparency coefficient adjustment and depth information and spatial positioning technology to provide real-time guidance and feedback for nursing operations, allowing users to intuitively see virtual guidance information such as operating steps, precautions and key skills in actual operations, thereby improving the accuracy and safety of operations. The adjustment of the transparency coefficient ensures that the virtual information will not completely obscure the real environment. The application of depth information and spatial positioning technology ensures the consistency of virtual content with the real environment, allowing virtual objects to be accurately aligned with objects in the real world, further enhancing the realism and intuitiveness of the operation.

[0102] Example 2

[0103] This embodiment is different from the previous one and provides an eye postoperative care simulation operating system based on mixed reality.

[0104] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0105] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0106] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0107] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A mixed reality-based eye postoperative care simulation operating system. Features: include, A simulation nursing platform module is used to establish a virtual nursing environment through the MR system; An operation simulation module, which is used to simulate the touch of real nursing operations through a force feedback handle and a tactile feedback glove; Motion capture module, used to capture the user's motion trajectory during postoperative care operations and process and analyze the data; The MR display terminal module is used to superimpose virtual postoperative care guidance information into the actual operating environment and provide instant operation guidance and feedback.

2. The mixed reality-based eye postoperative care simulation operating system as claimed in claim 1, Features: The MR system refers to a mixed reality system that integrates the real world and the virtual world by integrating hardware and software. The MR system includes MR glasses, tactile sensors, cameras, software platforms, laser pulse transmitters and detectors; The tactile sensor is used to collect tactile feedback data, and the camera is used to capture and pre-process the environment image in real time.

3. The mixed reality-based eye postoperative care simulation operating system as claimed in claim 2, Features: The establishment of a virtual care environment includes: Design virtual content according to the postoperative care process and write virtual content design documents, including the specific steps, precautions and virtual interaction requirements for each nursing task. The design documents include the visual effects, animation process and interaction methods of the virtual content; Use 3D modeling software to create 3D models related to postoperative care, including eye structures, care tools and environments, and create corresponding animations for each care task to show the correct operation steps and gestures; Optimize the details and texture of the 3D model, adjust the smoothness and realism of the animation, integrate the designed virtual content into the MR system software platform, collect user experience and feedback, and optimize and adjust the 3D modeling based on the feedback. The users refer to patients and their families; Users select postoperative care scenarios through the virtual interface in the MR glasses, including eye sealing care, wound cleaning, eye drops and functional exercises. By selecting the type of postoperative care scenario, the corresponding virtual care scene is loaded, including a nursing bed, facial model and care tools.

4. The mixed reality-based eye postoperative care simulation operating system as claimed in claim 3, Features: The tactile sense of simulating real nursing operations through the force feedback handle and the tactile feedback gloves refers to providing tactile feedback by using the force feedback handle and the tactile feedback gloves; Select a force feedback handle with six degrees of freedom, including three-axis translation and three-axis rotation, and select a high-precision brushless DC motor as the driving source of the feedback handle, and transmit the force feedback data to the central processor through a high-speed serial bus; Multi-point tactile feedback gloves are selected, and pressure point actuators are used as tactile feedback units. A tactile feedback unit is equipped on each finger and palm area. The tactile feedback unit simulates touch through vibration and pressure feedback. A distributed control system is used to independently control each tactile feedback unit. The distributed control system adjusts the tactile feedback in real time according to the user's operation, and transmits the tactile feedback data to the central processing unit through the wireless communication module.

5. The mixed reality-based eye postoperative care simulation operating system as claimed in claim 4, Features: Capturing the motion trajectory of the user during the postoperative care operation refers to using laser pulse emission and detection technology to capture the motion trajectory of the user in the force feedback handle and the tactile feedback glove; Install a laser pulse transmitter to cover the entire operation area with the emitted laser, and install a laser pulse detector on the force feedback handle and the tactile feedback glove; Locate the laser pulse signal, set the laser pulse transmitter emission frequency to 30Hz, 40Hz and 50Hz respectively, and record the time t when each transmitter emits the pulse signal i , use the laser pulse detector to receive the laser pulse signal from the transmitter and record the receiving time t r , according to the emission and reception time, the laser pulse signal propagation time T is calculated, the formula is: T=t r -t i Where T represents the propagation time of the laser pulse signal, t r Represents the laser pulse signal receiving time, t i Represents the time when each transmitter emits a laser pulse signal; Using the speed of light c, calculate the distance d from the detector to each emitter i , the formula is: d i =c·T Among them, d i represents the distance from the detector to each emitter, c represents the speed of light, and T represents the propagation time of the laser pulse signal; The detector position data is obtained according to the calculated distance data; Use laser pulse emitters and detectors to capture the movement trajectory of the user's hand and guide the user to perform hand movements through the system interface.

6. The mixed reality-based eye postoperative care simulation operating system as claimed in claim 5, Features: The data processing and analysis refers to transmitting the calculated detector position data to the central processing unit, which performs real-time analysis and processing of the motion trajectory after receiving the data, and generates real-time feedback and guidance information of the user operation; The analyzed and processed results are displayed to the user in real time through MR glasses, providing operation guidance and correction information. The system continuously tracks user operations and updates location data in real time.

7. The mixed reality-based eye postoperative care simulation operating system according to claim 6, Features: The method of using a camera to capture the environment image in real time and preprocessing the image includes: Users practice eye care operations in a virtual care environment, using a high-resolution camera to capture images of the user's surroundings in real time; The collected environmental images are preprocessed, including denoising, contrast enhancement and color correction. The preprocessed environmental images are transmitted to the MR glasses for image enhancement. The image enhancement algorithm is applied. The formula is: E=β·R+γ Wherein, E represents the enhanced image, R represents the original image, β represents the contrast adjustment coefficient, and γ represents the brightness adjustment value.

8. The mixed reality-based eye postoperative care simulation operating system as claimed in claim 7, Features: The virtual postoperative care guidance information is superimposed on the actual operation environment and providing instant operation guidance and feedback includes: According to the nursing process, the corresponding virtual nursing guidance information is generated, and the transparency coefficient is used to set the transparency of the virtual nursing guidance information. The formula is: I MR =I Real +α·I Virtual Among them, I MR Represents a mixed reality image, I Real represents a real-world image, I Virtual represents a virtual image, α is the transparency coefficient; Virtual care guidance information is superimposed on the enhanced environmental image for fusion, and objects in the virtual content are aligned with objects in the actual environment using depth information and spatial positioning technology.

9. A computer device, include: Memory and processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, the steps of the mixed reality-based postoperative eye care simulation operating system described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, Features: When the computer program is executed by a processor, the steps of the mixed reality-based postoperative eye care simulation operating system described in any one of claims 1 to 8 are implemented.

Images