A mixed reality-based simulation operation system for postoperative eye care

Abstract

The present invention discloses a mixed-reality-based simulation operation system for postoperative eye care, which relates to the technical field of postoperative care simulation. It includes a simulation care platform module for establishing a virtual care environment through an MR system; an operation simulation module for simulating the touch of real care operations through a force feedback handle and a tactile feedback glove; and a motion capture module for capturing the motion trajectory of the user during postoperative care operations and processing and analyzing the data. The present invention uses MR technology to establish a virtual care environment, enabling users to train in an environment close to reality. By using a force feedback handle and a tactile feedback glove, users can obtain a real operation touch. The motion capture module captures the user's operation trajectory to improve operation skills. By real-time collecting and processing images, virtual care guidance information is superimposed on the actual operation environment to provide instant guidance and feedback to users.

Description

Technical Field

[0001] The present invention relates to the technical field of postoperative care simulation, and particularly to a mixed-reality-based ocular postoperative care simulation operation system. Background Art

[0002] In recent years, mixed reality technology has developed rapidly and achieved remarkable results in multiple fields. Mixed reality technology combines augmented reality and virtual reality, captures images of the real world through an optical lens, performs image processing and enhancement through a processor, and then presents the optimized images in an almost latency-free manner. This technology has gradually gained attention in the field of healthcare. Especially in the postoperative care of ophthalmic surgery, mixed reality technology provides a safe and intuitive simulation environment for users and their families to better master nursing skills and reduce the incidence of postoperative complications.

[0003] Traditional ocular postoperative care mainly relies on the guidance of medical staff and paper or video teaching materials. However, these methods have significant deficiencies. Although video teaching can visually display nursing steps, it lacks interactivity, and it is difficult for users and their families to improve their proficiency through practice. Medical staff often face heavy work pressure in clinical practice and are difficult to provide sufficient time for detailed postoperative care guidance. Due to the lack of professional knowledge of users and their families, they may encounter problems such as eyelashes scratching the eyeball, incomplete closure of eye ointment, and blood scab accumulation at the suture site during actual care. This not only affects postoperative recovery but may also lead to serious complications such as exposure keratitis. Summary of the Invention

[0004] In view of the problems existing in the existing mixed-reality-based ocular postoperative care simulation operation system, the present invention is proposed.

[0005] Therefore, the problems to be solved by the present invention are that the existing video teaching and paper guidance materials lack interactivity, it is difficult for users and their families to effectively master nursing skills through these methods, medical staff resources are strained, and it is impossible to provide personalized and detailed nursing guidance for each user and their family. Due to the lack of practical experience during the nursing process, users and their families are prone to operation errors, increasing the risk of postoperative complications.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: A mixed-reality-based ocular postoperative care simulation operation system, which includes,

[0007] A simulation nursing platform module for establishing a virtual nursing environment through an MR system;

[0008] An operation simulation module for simulating the touch of real nursing operations through a force feedback handle and a haptic feedback glove;

[0009] A motion capture module, which is used to capture the motion trajectory of the user during postoperative care operations and process and analyze the data;

[0010] An MR display terminal module, which is used to superimpose virtual postoperative care guidance information onto the actual operation environment and provide instant operation guidance and feedback.

[0011] As a preferred solution of the simulated operating system for eye postoperative care based on mixed reality according to the present invention, wherein: the MR system refers to a mixed reality system, which integrates hardware and software to fuse the real world and the virtual world. The MR system includes an MR glasses, a tactile sensor, a camera, a software platform, a laser pulse emitter and a detector;

[0012] The tactile sensor is used to collect tactile feedback data, and the camera is used to capture the environmental image in real time and perform preprocessing.

[0013] As a preferred solution of the simulated operating system for eye postoperative care based on mixed reality according to the present invention, wherein: the establishment of the virtual care environment includes designing virtual content according to the postoperative care process, and compiling a virtual content design document, including the specific steps, precautions and virtual interaction requirements of each care task. The design document includes the visual effects, animation process and interaction methods of the virtual content;

[0014] Use 3D modeling software to create 3D models related to postoperative care, including eye structures, care tools and environments, and make corresponding animations for each care task to show the correct operation steps and gestures;

[0015] Optimize the details and textures of the 3D models, adjust the smoothness and authenticity of the animations, integrate the designed virtual content onto the MR system software platform, collect user experiences and feedback, and optimize and adjust the 3D modeling according to the feedback;

[0016] The user selects postoperative care scenarios through the virtual interface in the MR glasses, including eye sealing care, wound cleaning, eye drop instillation and functional exercise. By selecting the type of postoperative care scenario, the corresponding virtual care scene is loaded, including a nursing bed, a face model and care tools.

[0017] As a preferred solution of the simulated operating system for eye postoperative care based on mixed reality according to the present invention, wherein: the simulation of the touch feeling of real care operations by the force feedback handle and the tactile feedback glove means using the force feedback handle and the tactile feedback glove to provide tactile feedback;

[0018] Select a force feedback handle with six degrees of freedom, and the six degrees of freedom include three-axis translation and three-axis rotation. Select a high-precision brushless DC motor as the drive source of the feedback handle, and transmit the force feedback data to the central processor through a high-speed serial bus;

[0019] Select a multi-point tactile feedback glove, use a pressure point actuator as the tactile feedback unit, and equip each finger and palm area with a tactile feedback unit. The tactile feedback unit simulates the sense of touch through vibration and pressure feedback. A distributed control system is used to independently control each tactile feedback unit. The distributed control system adjusts the tactile feedback in real time according to the user's operation, and transmits the tactile feedback data to the central processor through a wireless communication module.

[0020] As a preferred solution of the post-operative eye care simulation operation system based on mixed reality according to the present invention, wherein: capturing the movement trajectory of the user during the post-operative care operation means using laser pulse emission and detection technology to capture the movement trajectory of the user in the force feedback handle and the tactile feedback glove;

[0021] Install a laser pulse emitter to cover the entire operation area with the emitted laser, and install laser pulse detectors on the force feedback handle and the tactile feedback glove;

[0022] Locate the laser pulse signal, set the emission frequencies of the laser pulse emitter to be 30Hz, 40Hz, and 50Hz respectively, and record the time t when each emitter emits a pulse signal i , use the laser pulse detector to receive the laser pulse signal from the emitter, and record the reception time t r , according to the emission and reception times, calculate the propagation time T of the laser pulse signal. The formula is:

[0023] T = t r - t i

[0024] wherein, T represents the propagation time of the laser pulse signal, t r represents the reception time of the laser pulse signal, and t i represents the time when each emitter emits a laser pulse signal;

[0025] Use the speed of light c to calculate the distance d from the detector to each emitter i , and the formula is:

[0026] d i = c·T

[0027] wherein, d i represents the distance from the detector to each emitter, c represents the speed of light, and T represents the propagation time of the laser pulse signal;

[0028] Obtain the detector position data based on the calculated distance data;

[0029] Use the laser pulse emitter and detector to capture the movement trajectory of the user's hand, and guide the user to perform hand movements through the system interface.

[0030] As a preferred solution of the post - operative eye care simulation operation system based on mixed reality according to the present invention, wherein: the processing and analyzing of data means transmitting the calculated detector position data to the central processing unit. After receiving the data, the central processing unit performs real - time analysis and processing of the movement trajectory, and generates real - time feedback and guidance information for the user's operation;

[0031] The result of the analysis and processing is displayed to the user in real - time through the MR glasses, providing operation guidance and correction information. The system continuously tracks the user's operation and updates the position data in real - time.

[0032] As a preferred solution of the post - operative eye care simulation operation system based on mixed reality according to the present invention, wherein: the real - time shooting and pre - processing of the environmental image using the camera includes,

[0033] The user performs eye care operation practice in the virtual care environment, and uses a high - resolution camera to capture the environmental image around the user in real - time;

[0034] Pre - process the collected environmental image, including denoising, enhancing contrast, and color correction, and transmit the pre - processed environmental image to the MR glasses for image enhancement. Apply the image enhancement algorithm, and the formula is:

[0035] E = β·R + γ

[0036] Wherein, E represents the enhanced image, R represents the original image, β represents the contrast adjustment coefficient, and γ represents the brightness adjustment value.

[0037] As a preferred solution of the post - operative eye care simulation operation system based on mixed reality according to the present invention, wherein: the overlaying of the virtual post - operative care guidance information onto the actual operation environment and providing immediate operation guidance and feedback includes,

[0038] Generate corresponding virtual care guidance information according to the care process, and set the transparency of the virtual care guidance information using the transparency coefficient. The formula is:

[0039] I MR =I Real +α·I Virtual

[0040] Wherein, I MR represents the mixed reality image, I Real represents the real - world image, I Virtual represents the virtual image, and α is the transparency coefficient;

[0041] Overlay the virtual care guidance information onto the enhanced environmental image for fusion, and use depth information and spatial positioning technology to align the objects in the virtual content with the objects in the actual environment.

[0042] A computer device includes: a memory and a processor; the memory stores a computer program, and when the processor executes the computer program, steps of a mixed reality-based eye postoperative care simulation operation system are implemented.

[0043] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, steps of a mixed reality-based eye postoperative care simulation operation system are implemented.

[0044] The beneficial effects of the present invention are as follows: The present invention uses MR technology to establish a virtual care environment to enable users to train in an environment close to reality, adopts a force feedback handle and a tactile feedback glove to enable users to obtain a real operation touch feeling, a motion capture module captures the user's operation trajectory to improve operation skills, and by collecting and processing images in real time, virtual care guidance information is superimposed on the actual operation environment to provide instant guidance and feedback to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0046] Figure 1 It is a schematic structural diagram of a mixed reality-based eye postoperative care simulation operation system.

[0047] Figure 2 It is a schematic flowchart of a mixed reality-based eye postoperative care simulation operation system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the drawings in the specification.

[0049] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention, but the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0050] Second, the "one embodiment" or "embodiment" referred to herein means a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or selectively exclusive embodiments from other embodiments.

[0051] Embodiment 1

[0052] Referring to Figure 1 and Figure 2 , which is the first embodiment of the present invention. This embodiment provides a mixed reality-based postoperative eye care simulation operation system. A mixed reality-based postoperative eye care simulation operation system includes:

[0053] S1. A simulation care platform module for establishing a virtual care environment through an MR system;

[0054] Specifically, the MR system refers to a mixed reality system that integrates hardware and software to fuse the real world and the virtual world. The MR system includes an MR glasses, a tactile sensor, a camera, a software platform, a laser pulse emitter, and a detector;

[0055] Use the tactile sensor to collect tactile feedback data, and use the camera to capture environmental images in real time and perform preprocessing.

[0056] The MR system can provide an immersive and interactive experience. By fusing the real world and the virtual world, it enhances the user's perception and operation ability of information. By integrating high-performance hardware and advanced software, it can provide users with a brand-new and immersive mixed reality experience.

[0057] Furthermore, establishing a virtual care environment includes,

[0058] Design virtual content according to the postoperative care process, and compile a virtual content design document, including the specific steps, precautions, and virtual interaction requirements of each care task. The design document includes the visual effects, animation process, and interaction methods of the virtual content;

[0059] Steps of washing hands: Turn on the faucet, moisten both hands with warm water, take hand sanitizer, rub hands for at least 20 seconds, cover all parts, rinse hands thoroughly with clean water, dry hands with a clean towel, and use a tissue when turning off the faucet;

[0060] Eye Sealing Care Steps: The user wears an MR glasses, and virtual handwashing steps are presented in the user's field of vision. After the user finishes washing hands, the user selects eye ointment through gestures or voice commands. The MR system simulates checking the expiration date and packaging integrity of the eye ointment. When the user is operating, the MR system displays the distance between the eye ointment nozzle and the eyeball in real time, and uses colors and sounds to prompt the user to adjust to the optimal distance. The MR system displays a virtual eyeball model in the user's field of vision and demonstrates pulling down the lower eyelid and applying the eye ointment. The user follows the virtual guidance to simulate applying the eye ointment. The MR system monitors the user's gestures and operations in real time, collects feedback and adjustment suggestions. The MR system displays virtual models of sterile gauze and 3M transparent tape in the user's field of vision. The MR system demonstrates the process of covering the eye and fixing the gauze and tape sticking and adjusts the tightness of the tape in real time;

[0061] Eye Ointment Application Steps: Select the eye ointment according to the treatment needs, open the package of the sterile cotton swab, take out the cotton swab and place it on a clean sterile workbench for standby. Use the sterile cotton swab to push aside the eyelashes and check if there is dirt around the eyelashes. If there is dirt, gently wipe it clean with the sterile cotton swab. Gently pull down the user's lower eyelid with your hand to expose the conjunctival sac. Use the other hand to squeeze out an appropriate amount of eye ointment and evenly apply it inside the lower eyelid. Relax the eyelid and gently close the eyes. Take a piece of sterile gauze and cover the eye. Use medical tape to start pasting from around the eyelid to fix the gauze;

[0062] Wound Cleaning Steps: The MR system automatically calculates the dosage of the disinfectant according to the user's requirements, monitors the user's operation strength and time through sensors and gives real-time feedback and adjustment suggestions. The user performs handwashing operations according to the instructions in the virtual environment. Prepare a bottle of 0.5% concentration povidone iodine disinfection solution and sterile cotton swabs. Dip the sterile cotton swab into the povidone iodine solution and gently wipe around the wound from the inside out. Prepare a bottle of 0.9% physiological saline. Use a dropper to slowly drip the saline from the inner canthus to the outer canthus to clean the conjunctiva. Check the wound. If there is a foreign body, carefully remove it with sterile forceps and gently dry the eye area;

[0063] Eye Drop Instillation Steps: The user performs handwashing operations according to the instructions in the virtual environment. The MR system prompts the user to select sodium hyaluronate eye drops and shows the appearance and correct usage method of sodium hyaluronate eye drops. The MR system prompts the user to keep the dropper of the eye drop bottle 2 cm away from the eyeball. In the virtual environment, the user can move the eye drop bottle to the specified distance through hand operations and see the distance scale in the MR glasses in real time. Under the prompt of the MR system, the user gently pulls down the lower eyelid with one hand, and holds the eye drop bottle 2 cm above the other hand, gently squeezes the bottle body, and drops the eye drops into the small pocket formed by the lower eyelid. The MR system prompts the user to gently close the eyes and turn the eyeball in all directions, and tracks the eyeball movement in real time and gives visual feedback;

[0064] Functional exercise steps: The user wears an MR glasses and tries to focus on a virtual sphere by moving the head and eyes. The MR system monitors the user's focusing degree in real time and displays the moving path of a sphere in the virtual environment, including up, down, left, right and the middle positions, and shows clear indication signs to guide the direction of the user's eye movement;

[0065] Use 3D modeling software to create 3D models related to postoperative care, including eye structures, nursing tools and environments, and make corresponding animations for each nursing task to demonstrate the correct operation steps and gestures;

[0066] Optimize the details and textures of the 3D models, adjust the smoothness and authenticity of the animations, integrate the designed virtual content into the MR system, collect user experiences and feedback, and optimize and adjust the 3D modeling according to the feedback;

[0067] The user selects postoperative care scenarios through the virtual interface in the MR glasses, including eye sealing care, wound cleaning, eye drop instillation and functional exercise. By selecting the type of postoperative care scenario, the corresponding virtual care scene is loaded, including a nursing bed, a user's face model and nursing tools.

[0068] In the virtual environment, using 3D models and animation processes, the user can practice repeatedly in the virtual environment, better master nursing skills. Through the MR system, the user can clearly see each step and the correct gestures, be immersed in the virtual environment of postoperative care, which helps to improve the accuracy of operations, deepen the user's understanding and memory of the nursing process. By collecting user experiences and feedback, timely adjust and optimize the 3D models and animations to ensure the quality and applicability of the teaching content. The development and use of this virtual nursing content will promote technological innovation in the field of medical care and contribute to the popularization and application of advanced technologies in the medical industry.

[0069] S2. An operation simulation module, which is used to simulate the touch feeling of real nursing operations through a force feedback handle and a tactile feedback glove;

[0070] Specifically, simulating the touch feeling of real nursing operations through a force feedback handle and a tactile feedback glove means using the force feedback handle and the tactile feedback glove to provide touch feedback;

[0071] Select a force feedback handle with six degrees of freedom, where the six degrees of freedom include three-axis translation and three-axis rotation. Select a high-precision brushless DC motor as the drive source of the feedback handle, and transmit the force feedback data to the central processor through a high-speed serial bus;

[0072] Select a multi-point tactile feedback glove, use the pressure point actuator as the tactile feedback unit, and equip each finger and palm area with a tactile feedback unit. The tactile feedback unit simulates the sense of touch through vibration and pressure feedback. A distributed control system is used to independently control each tactile feedback unit. The distributed control system adjusts the tactile feedback in real time according to the user's operation situation, and transmits the tactile feedback data to the central processor through a wireless communication module.

[0073] The force feedback handle and the tactile feedback glove can simulate the strength and touch in real nursing operations, enabling users to adjust their operations in a timely manner during the training process. This helps users master the correct strength and techniques during the operation, improve the accuracy of nursing operations, and provide data support for teaching and research by recording the operation data of users during the training process, which is helpful for improving the training effect of nursing operation skills.

[0074] S3. A motion capture module, which is used to capture the motion trajectory of the user during postoperative nursing operations and process and analyze the data;

[0075] Specifically, capturing the motion trajectory of the user during postoperative nursing operations means using laser pulse emission and detection technology to capture the motion trajectory of the user in the force feedback handle and the tactile feedback glove;

[0076] Install a laser pulse emitter so that the emitted laser covers the entire operation area, and install laser pulse detectors on the force feedback handle and the tactile feedback glove;

[0077] Locate the laser pulse signal, set the emission frequencies of the laser pulse emitters to be 30Hz, 40Hz, and 50Hz respectively, and record the time t when each emitter emits a pulse signal i , use the laser pulse detector to receive the laser pulse signal from the emitter, and record the reception time t r , according to the emission and reception times, calculate the propagation time T of the laser pulse signal. The formula is:

[0078] T = t r -t i

[0079] Among them, T represents the propagation time of the laser pulse signal, t r represents the reception time of the laser pulse signal, t i represents the time when each emitter emits a laser pulse signal;

[0080] Use the speed of light c to calculate the distance d from the detector to each emitter i , the formula is:

[0081] d i = c·T

[0082] Among them, di where \(d\) represents the distance from the detector to each emitter, \(c\) represents the speed of light, and \(T\) represents the propagation time of the laser pulse signal;

[0083] Obtain the detector position data based on the calculated distance data;

[0084] Use a laser pulse emitter and a detector to capture the movement trajectory of the user's hand, and guide the user to perform hand movements through the system interface.

[0085] Laser pulse technology can provide high-precision positioning information to ensure the accuracy of the user's hand movement trajectory. By capturing the user's hand movement trajectory in real time, the MR system can immediately provide feedback to help the user understand whether their operation is accurate and make timely adjustments, thereby improving the training effect. By accurately capturing the hand movement trajectory, the system can provide a more realistic and intuitive training experience for the user, thereby improving the training quality. The system can record the user's movement trajectory data, which can be used for subsequent analysis and evaluation. Laser pulse technology can adapt to different operating environments and regions, with strong flexibility and adaptability.

[0086] Furthermore, the data processing and analysis refers to transmitting the calculated detector position data to the central processing unit. After receiving the data, the central processing unit performs real-time analysis and processing of the movement trajectory to generate real-time feedback and guidance information for the user's operation;

[0087] The results of the analysis and processing are displayed to the user in real time through the MR glasses to provide operation guidance and correction information. The system continuously tracks the user's operation and updates the position data in real time.

[0088] By displaying the results of the analysis and processing to the user in real time, the system can provide immediate operation guidance and correction information to help the user understand whether their operation is accurate in a timely manner during the training process and make corresponding adjustments. The processing and analysis of real-time data can provide a more realistic and dynamic training environment, making the training closer to the actual working scenario, thereby enhancing the training effect. The MR system can adjust the feedback information in real time according to the user's operation situation to provide a personalized training plan. At the same time, the system can also automatically adjust the training difficulty according to the user's progress to achieve adaptive training.

[0089] Furthermore, the use of a camera to capture environmental images in real time and perform preprocessing includes,

[0090] The user performs eye care operation exercises in a virtual care environment, and uses a high-resolution camera to capture the environmental images around the user in real time;

[0091] Preprocess the collected environmental images, including denoising, enhancing contrast, and color correction, and transmit the preprocessed environmental images to the MR glasses for image enhancement. Apply the image enhancement algorithm, the formula is:

[0092] E = β·R + γ

[0093] Among them, E represents the enhanced image, R represents the original image, β represents the contrast adjustment coefficient, and γ represents the brightness adjustment value.

[0094] Real-time capture and preprocess the environmental images around the user through a high-resolution camera, and then use the MR glasses for image enhancement. This process brings significant benefits to virtual nursing operation training. It not only improves the realism of the operation and the clarity of visual feedback, enhances the training effect, but also ensures the safety of the training. The virtual training method that combines real-time image acquisition and processing provides users with an efficient, safe, and highly interactive training solution.

[0095] S4. MR display terminal module, used to overlay virtual postoperative care guidance information onto the actual operation environment and provide immediate operation guidance and feedback;

[0096] Specifically, overlaying virtual care guidance information onto the actual operation environment and providing immediate operation guidance and feedback includes,

[0097] Generate corresponding virtual care guidance information according to the care process, and set the transparency of the virtual care guidance information using the transparency coefficient. The formula is:

[0098] I MR = I Real + α·I Virtual

[0099] Among them, I MR represents the mixed reality image, I Real represents the real-world image, I Virtual represents the virtual image, and α is the transparency coefficient;

[0100] Overlay the virtual care guidance information onto the real-time collected environmental images for fusion. Use depth information to determine the position of the virtual content in the three-dimensional space and its relative position relationship with real-world objects. Use spatial positioning technology to enable the MR system to real-time track the position changes of the user and the device and adjust the position and perspective of the virtual content, and align the objects in the virtual content with the objects in the actual environment.

[0101] The virtual nursing guidance information is superimposed on the actual operation environment and integrated through the adjustment of the transparency coefficient and depth information and spatial positioning technology, providing real-time guidance and feedback for nursing operations. It allows users to intuitively see the virtual guidance information, such as operation steps, precautions, and key skills, during actual operations, thereby improving the accuracy and safety of operations. The adjustment of the transparency coefficient ensures that the virtual information does not completely block the real environment, and the application of depth information and spatial positioning technology guarantees the consistency between the virtual content and the real environment, enabling virtual objects to accurately align with objects in the real world, further enhancing the realism and intuitiveness of operations.

[0102] Embodiment 2

[0103] This embodiment is different from the previous one and provides a mixed reality-based simulation operation system for postoperative eye care.

[0104] If the described functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical disks, etc., which can store program codes.

[0105] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0106] More specific examples (a non-exhaustive list) of computer-readable media include the following: electrical connections (electronic devices) having one or more wirings, portable computer diskettes (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0107] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having suitable combinational logic gate circuits, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0108] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A mixed reality-based simulation operation system for postoperative eye care, characterized in that: including, a simulation nursing platform module for establishing a virtual nursing environment through an MR system; an operation simulation module for simulating the touch of real nursing operations through a force feedback handle and a tactile feedback glove; a motion capture module for capturing the motion trajectory of a user during postoperative nursing operations and processing and analyzing the data; an MR display terminal module for superimposing virtual postoperative nursing guidance information onto the actual operation environment and providing instant operation guidance and feedback; the MR system refers to a mixed reality system that integrates hardware and software to fuse the real world and the virtual world. The MR system includes an MR glasses, a tactile sensor, a camera, a software platform, a laser pulse emitter, and a detector; using the tactile sensor to collect tactile feedback data and using the camera to capture and preprocess environmental images in real time; the establishment of the virtual nursing environment includes, designing virtual content according to the postoperative nursing process and preparing a virtual content design document, including the specific steps, precautions, and virtual interaction requirements of each nursing task. The design document includes the visual effects, animation process, and interaction methods of the virtual content; using 3D modeling software to create 3D models related to postoperative nursing, including eye structures, nursing tools, and environments, and making corresponding animations for each nursing task to demonstrate the correct operation steps and gestures; optimizing the details and textures of the 3D models, adjusting the smoothness and authenticity of the animations, integrating the designed virtual content into the MR system software platform, collecting user experiences and feedback, and optimizing and adjusting the 3D modeling according to the feedback. The user refers to patients and their families; the user selects a postoperative nursing scenario through the virtual interface in the MR glasses, including eye sealing care, wound cleaning, eye drop instillation, and functional exercise. By selecting the type of postoperative nursing scenario, the corresponding virtual nursing scene is loaded, including a nursing bed, a face model, and nursing tools.

2. The mixed reality-based simulation operation system for postoperative eye care according to claim 1, characterized in that: the simulation of the touch of real nursing operations through the force feedback handle and the tactile feedback glove means using the force feedback handle and the tactile feedback glove to provide tactile feedback; selecting a force feedback handle with six degrees of freedom, where the six degrees of freedom include three-axis translation and three-axis rotation, selecting a high-precision brushless DC motor as the drive source of the feedback handle, and transmitting the force feedback data to the central processor through a high-speed serial bus; selecting a multi-point tactile feedback glove, using a pressure point actuator as the tactile feedback unit, and equipping a tactile feedback unit in each finger and palm area. The tactile feedback unit simulates the touch through vibration and pressure feedback, and a distributed control system is used to independently control each tactile feedback unit. The distributed control system adjusts the tactile feedback in real time according to the user's operation situation, and transmits the tactile feedback data to the central processor through a wireless communication module.

3. The mixed reality-based simulation operation system for postoperative eye care according to claim 2, characterized in that: the capture of the user's motion trajectory during postoperative nursing operations refers to using laser pulse emission and detection technology to capture the motion trajectory of the user in the force feedback handle and the tactile feedback glove; installing a laser pulse emitter to cover the entire operation area with the emitted laser, and installing laser pulse detectors on the force feedback handle and the tactile feedback glove; Locate the laser pulse signal, set the emission frequencies of the laser pulse emitters to 30 Hz, 40 Hz, and 50 Hz respectively, and record the time t when each emitter emits a pulse signal i , use a laser pulse detector to receive the laser pulse signal from the emitter, and record the reception time t r , calculate the propagation time T of the laser pulse signal based on the emission and reception times. The formula is: T = tr - ti Among them, T represents the propagation time of the laser pulse signal, and t r represents the reception time of the laser pulse signal, and t i represents the time when each transmitter emits the laser pulse signal; Calculate the distance d from the detector to each transmitter using the speed of light c i , and the formula is: di = c · T Among them, d i represents the distance from the detector to each transmitter, c represents the speed of light, and T represents the propagation time of the laser pulse signal; obtaining the detector position data based on the calculated distance data; Using a laser pulse emitter and a detector, capture the movement trajectory of the user's hand, and guide the user to perform hand movements through the system interface.

4. The mixed reality-based simulation operation system for postoperative eye care according to claim 3, characterized in that: The processing and analysis of the data refers to transmitting the calculated detector position data to the central processing unit. After receiving the data, the central processing unit performs real-time analysis and processing of the movement trajectory, generating real-time feedback and guidance information for the user's operation. The results of the analysis and processing are displayed to the user in real time through the MR glasses, providing operation guidance and correction information. The system continuously tracks the user's operation and updates the position data in real time.

5. The mixed reality-based simulation operation system for postoperative eye care according to claim 4, characterized in that: The real-time shooting and preprocessing of the environmental image using the camera includes The user performs eye care operation practice in the virtual care environment, and uses a high-resolution camera to capture the environmental image around the user in real time. Preprocess the collected environmental image, including denoising, enhancing contrast, and color correction. Transmit the preprocessed environmental image to the MR glasses for image enhancement, and apply the image enhancement algorithm. The formula is: E = β·R + γ Where, E represents the enhanced image, R represents the original image, β represents the contrast adjustment coefficient, and γ represents the brightness adjustment value.

6. The simulation operation system for postoperative eye care based on mixed reality according to claim 5, characterized in that: The overlaying of the virtual postoperative care guidance information onto the actual operation environment and providing immediate operation guidance and feedback includes Generate corresponding virtual care guidance information according to the care process, and set the transparency of the virtual care guidance information using the transparency coefficient. The formula is: I MR = I R eal + α·I V irtual It should be noted that the text seems to have some formatting or content issues that might make it less clear in terms of its exact meaning and intended use. The "eal" part in "eal + α·I" is rather unusual and might be a misspelling or something specific to a particular context that requires further clarification. Among them, I MR represents a mixed reality image, I Real represents a real-world image, I Virtual represents a virtual image, and α is the transparency coefficient; Overlay the virtual care guidance information onto the enhanced environmental image for fusion. Use depth information and spatial positioning technology to align the objects in the virtual content with the objects in the actual environment.

7. A computer device, comprising: A memory and a processor; The memory stores a computer program, characterized in that: when the processor executes the computer program, it implements the steps of the mixed reality-based eye postoperative care simulation operation system according to any one of claims 1 to 6.

8. A computer-readable storage medium, on which a computer program is stored, characterized in that: When the computer program is executed by the processor, it implements the steps of the mixed reality-based eye postoperative care simulation operation system according to any one of claims 1 to 6.

Images