A tactile-visual co-sensory simulation method for intravenous puncture in virtual training for intravenous therapy nurses
By using a layered collision detection and event triggering method, combined with tactile-visual collaborative feedback, the problems of single tactile feedback and lack of visual feedback in the virtual puncture training system are solved. This achieves a high degree of consistency between tactile and visual feedback during needle puncture, thereby improving training effectiveness and immersion.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SICHUAN UNIV
- Filing Date
- 2026-04-17
- Publication Date
- 2026-06-02
AI Technical Summary
Existing virtual puncture training systems have a single dimension of tactile feedback, which cannot accurately simulate the mechanical sensation of a needle tip penetrating different tissue layers. Visual feedback is lacking and the coordination between tactile and visual feedback is poor, resulting in trainees being unable to establish correct puncture feel memory and reduced immersion.
By employing layered collision detection and event triggering, combined with a tactile-visual collaborative feedback method, the arm and blood vessel models are modeled in layers to generate multi-layered tactile feedback and visual blood return signals. These signals are then synchronously output through the same event trigger signal to ensure a high degree of coordination between touch and vision.
It achieves a high degree of consistency between touch and vision during needle puncture, enhances the realism of puncture feel and operational recognition, provides dual confirmation, and improves training effectiveness and immersion.
Smart Images
Figure CN122135620A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical virtual simulation training technology, specifically to a tactile and visual feedback simulation method for nursing skills training. Background Technology
[0002] In recent years, with the rapid development of extended reality (XR) and force feedback technologies, virtual puncture training systems based on computer simulation and tactile interaction have gradually entered the field of nursing education. These systems construct a virtual human arm through 3D modeling and combine it with force feedback devices to simulate the mechanical changes during the puncture process, theoretically providing trainees with a repeatable, risk-free, and standardized training environment. However, existing virtual puncture training systems generally suffer from the following technical deficiencies in practical applications: (1) Single dimension of tactile feedback. Most current virtual puncture training systems can only provide simple resistance that increases linearly with puncture depth. They cannot accurately simulate the differentiated mechanical sensations generated when the needle tip penetrates the skin layer, the anterior wall layer of the blood vessel, enters the lumen layer of the blood vessel, and punctures the posterior wall layer of the blood vessel. In particular, the "feeling of emptiness" perceived by the hand when the puncture is successful in clinical practice—that is, the typical feeling of the resistance suddenly decreasing after the needle tip breaks through the anterior wall of the blood vessel—is almost impossible to reflect in the existing system, making it difficult for trainees to establish correct puncture feel memory.
[0003] (2) Lack of visual feedback on blood return. When the needle tip enters the blood vessel correctly, the blood will naturally flow back into the transparent needle tube to form a red liquid column because the pressure inside the blood vessel is higher than the pressure inside the needle tube. This visual signal occurs simultaneously with the feeling of emptiness in the hand, which together constitutes a double confirmation of successful puncture. However, existing virtual systems sometimes completely ignore the blood return simulation and lack a precise synchronization relationship with the actual position of the needle tip and tactile events.
[0004] (3) Poor coordination between tactile and visual senses. Because the collision detection module, force rendering module, and graphics rendering module are usually developed independently and run on different threads or at different refresh rates, there is a perceptible delay or logical inconsistency in the output of tactile and visual signals. For example, trainees may see the blood return animation before feeling the breakthrough, or the blood return may not appear for a long time after the breakthrough occurs. This sensory misalignment not only destroys the sense of immersion, but may also lead trainees to establish incorrect neuromuscular memories that contradict the sensory feedback patterns in real clinical operations, thus negatively impacting the training effect. Summary of the Invention
[0005] This invention addresses the technical problems of existing virtual training systems for intravenous puncture, such as limited tactile feedback, lack of visual blood return feedback, and poor coordination between tactile and visual feedback. It provides a tactile-visual collaborative realistic simulation method that enables trainees to simultaneously experience the dual realistic perceptions of "feeling emptiness" and "stable blood return within the needle" when puncturing and breaking through blood vessels, significantly improving the realism and operational recognition of virtual puncture training.
[0006] To achieve the above objectives, the present invention adopts the following technical solution: A tactile-visual collaborative realistic simulation method for intravenous puncture in virtual training for intravenous therapy nurses. This method comprises two parts: layered collision detection and event triggering, and tactile-visual collaborative feedback. Specifically, the layered collision detection and event triggering involves: Layered collision modeling is performed on the arm model and blood vessel model, which are divided into at least the skin layer, the anterior wall layer of the blood vessel, the lumen layer of the blood vessel, and the posterior wall layer of the blood vessel. Independent collision boundaries are set for each layer, so that the model has hierarchical identifiable attributes and real-time collision response capability. The system acquires the three-dimensional spatial coordinates of the virtual puncture needle tip in real time, determines the current tissue level of the needle tip, and generates corresponding event trigger signals when the needle tip crosses the boundaries of different levels. The events include "entry into the blood vessel lumen event" and "puncture of the posterior wall of the blood vessel event".
[0007] The tactile-visual co-feedback specifically refers to: Based on the tissue layer where the needle tip is located, multi-layered tactile feedback signals are generated. When the needle tip contacts the skin layer, it outputs a surface damping sensation and resistance with increasing stiffness. When the needle tip breaks through the anterior wall of the blood vessel and enters the blood vessel lumen, it outputs a distinct instantaneous breakthrough impact force and a feeling of emptiness. When the needle tip continues to advance and pierces the posterior wall of the blood vessel, the feeling of emptiness disappears, and only a low level of puncture resistance is maintained. Based on the tissue layer where the needle tip is located, a corresponding visual blood return feedback signal is generated. When it is determined that the needle tip has entered the vascular lumen layer, a red liquid column is generated inside the virtual needle tube, which rises steadily from the needle tip to the needle handle and remains there. When it is determined that the needle tip has pierced the posterior wall layer of the blood vessel, the red liquid column flashes briefly and then disappears immediately. The tactile feedback signal and the visual blood return feedback signal are output synchronously based on the same event trigger signal, so that the trainee can simultaneously obtain the dual real perception of "feeling of emptiness" and "stable blood return in the needle" when puncturing and breaking through the blood vessel, thereby achieving close coordination and sensory consistency between touch and vision at key operation nodes.
[0008] Preferably, in the multi-layered tactile feedback signal, the damping and resistance of the skin layer simulate the tightness of the skin when it is compressed; the instantaneous impact and sense of loss when the anterior wall of the blood vessel is broken simulate the abruptness of piercing the blood vessel wall and the physiological characteristics of the subsequent sudden drop in resistance.
[0009] Preferably, in the visual blood return feedback signal, a steadily rising and continuously maintained red liquid column simulates the blood return phenomenon after a successful clinical puncture, providing trainees with visual confirmation of success; a red liquid column that flashes briefly and then disappears simulates the abnormal phenomenon of interrupted blood return after puncturing a blood vessel, forming a dual error prompt with the change in tactile pressure.
[0010] Preferably, the synchronous output of tactile feedback signals and visual feedback signals adopts an event-driven parallel processing architecture, so that the two are output simultaneously based on the same event trigger signal, with the output time difference being less than the human perception threshold, thereby achieving complete consistency between touch and vision at the perception level, eliminating sensory misalignment, and enhancing immersion and realism.
[0011] Compared with the prior art, the present invention has the following beneficial effects: (1) The tactile feedback is rich in layers and highly realistic. By performing layered collision modeling of the skin layer, the anterior wall of the blood vessel, the lumen of the blood vessel, and the posterior wall of the blood vessel, it is possible to accurately simulate the mechanical changes when the needle tip passes through different tissues, especially the instantaneous breakthrough impact and the feeling of falling when entering the lumen of the blood vessel, so that trainees can obtain a highly realistic puncture feel and effectively establish correct muscle memory.
[0012] (2) The visual blood return feedback is accurate and logically consistent. Based on the actual level of the needle tip, a red liquid column is dynamically generated that rises steadily and remains or flashes and disappears momentarily, realistically representing the blood return performance under two different results: successful puncture and puncture of blood vessels. The visual signal is highly consistent with the real clinical scenario.
[0013] (3) Tactile and visual senses are highly coordinated. Both are triggered by the same event and output synchronously. The time difference between the outputs is less than the human perception threshold, which completely eliminates sensory misalignment. When the trainee punctures and breaks through the blood vessel, he can simultaneously feel the sense of emptiness and the stable blood return, forming a double confirmation, which greatly enhances the sense of immersion and training effect.
[0014] (4) Provides dual error prompts. When a trainee accidentally punctures the posterior wall of a blood vessel, the tactile sensation of loss disappears and the visual sensation of blood return flashes briefly before disappearing. Both senses simultaneously provide a failure prompt, helping trainees quickly identify and correct errors, significantly improving skill mastery efficiency. This invention can be widely applied in the field of virtual simulation training for nursing skills. Attached Figure Description
[0015] Figure 1 The flowchart of the present invention illustrates the logical relationship between the two main parts: layered collision detection and event triggering, and tactile-visual collaborative feedback.
[0016] Figure 2 This is a module architecture diagram of the method of the present invention in the Unity engine, showing the data flow and event-driven relationship between the collision detection module, event manager, haptic feedback module, and visual feedback module. Detailed Implementation
[0017] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention. This embodiment is developed based on the Unity game engine, uses the PICO series VR headset as the visual display device, and cooperates with a force feedback device to achieve tactile-visual collaborative simulation of vein puncture.
[0018] like Figure 1 As shown, the overall process of the method of this invention is divided into two main stages: layered collision detection and event triggering, and tactile-visual collaborative feedback. First, the steps of the layered collision detection and event triggering stage are executed. Layered collision modeling (S1) is performed on the arm model and the blood vessel model, specifically including: S101, adding colliders to the skin layer of the arm model and setting boundaries to give it skin-layer identifiable attributes; S102, adding colliders to the anterior wall region of the blood vessel and setting the boundaries of the anterior wall layer; S103, creating a tubular hollow collider inside the blood vessel and setting the boundaries of the blood vessel lumen layer, with its inner diameter consistent with the inner diameter of the blood vessel; S104, adding colliders to the posterior wall region of the blood vessel and setting the boundaries of the posterior wall layer. Through the above steps, the model possesses at least layered identifiable attributes and real-time collision response capabilities for the skin layer, anterior wall layer, blood vessel lumen layer, and posterior wall layer.
[0019] After completing the layered collision modeling, the step of real-time acquisition of needle tip coordinates and event triggering is executed (S2). S201, the collision detection script on the virtual puncture needle model reads the three-dimensional spatial coordinates of the needle tip every frame; S202, when it is detected that the previous frame is the anterior wall layer of the blood vessel and the current frame is the lumen layer of the blood vessel, the "entering the lumen of the blood vessel event" is triggered and the event signal is sent to the global event manager; S203, when it is detected that the previous frame is the lumen layer of the blood vessel and the current frame is the posterior wall layer of the blood vessel or exceeds the posterior wall range of the blood vessel, the "puncture the posterior wall of the blood vessel event" is triggered and the event signal is also sent to the global event manager.
[0020] After the event is triggered, the tactile-visual coordinated feedback stage begins. When the "entry into the blood vessel lumen event" occurs, tactile feedback and visual blood return feedback are output synchronously based on the same event signal. Specifically, the tactile feedback generation step (S3) is executed: S301, when the needle tip is in the skin layer, the stiffness coefficient of the force feedback is linearly increased according to the puncture depth and damping sensation is added, outputting surface damping sensation and resistance with increasing stiffness; S302, when the "entry into the blood vessel lumen event" is triggered, the tactile feedback module immediately generates an instantaneous breakthrough impact force, and then quickly reduces the force feedback amplitude to a low level, outputting an instantaneous breakthrough impact force and a sense of loss; S303, when the "puncture of the posterior wall of the blood vessel event" is triggered, the tactile feedback module further reduces the low-level resistance to the basic puncture resistance, while canceling the sense of loss state, maintaining only a low level of puncture resistance.
[0021] Simultaneously, the visual blood return feedback generation step (S4) is executed: S401, when the "entering blood vessel lumen event" is triggered, the visual blood return module starts the red liquid column rising animation inside the virtual needle tube, so that the red liquid column extends from the needle tip to the needle handle until it stabilizes, and then maintains this state to simulate the blood return phenomenon after successful clinical puncture; S402, when the "piercing blood vessel posterior wall event" is triggered, the visual blood return module starts the red liquid column disappearing animation, so that the red liquid column flashes for a moment and then disappears quickly, simulating the abnormal behavior of blood return interruption after piercing blood vessel.
[0022] Finally, the synchronization output step (S5) is executed. The event manager adopts an event-driven parallel processing architecture. In S501, when the "entry into blood vessel lumen event" or "puncture of blood vessel posterior wall event" is triggered, the event manager sequentially calls the callback functions registered by the tactile feedback module and the visual blood return module within the same frame, and outputs the tactile feedback signal and the visual blood return feedback signal synchronously based on the same event trigger signal; in S502, since the callback functions are executed sequentially within the same frame, the output time difference between the tactile signal and the visual signal is lower than the human perception threshold, thereby achieving complete consistency between touch and vision at the perception level and eliminating sensory misalignment.
[0023] In actual training, trainees wear a PICO head-mounted display and hold a force feedback device to perform vein punctures on a virtual arm model. When the needle tip correctly pierces the blood vessel, the trainee simultaneously feels an instantaneous breakthrough impact and a sudden drop in resistance (a feeling of emptiness) from their hand, and sees a stable red column of liquid rising and holding within the needle tube through the head-mounted display, forming a double confirmation of successful puncture. If the needle tip continues to advance and pierces the posterior wall of the blood vessel, the feeling of emptiness immediately disappears, and the red column of liquid flashes and then disappears. Based on this, the trainee judges the operation as a failure and adjusts the puncture depth. The above process fully demonstrates the close coordination and sensory consistency of tactile and visual sensations at key operational points in this invention.
Claims
1. A method for tactile-visual collaborative realism simulation of intravenous puncture for virtual training of intravenous therapy nurses, characterized in that... This method includes two parts: layered collision detection and event triggering, and haptic-visual collaborative feedback. The layered collision detection and event triggering method is specifically described as follows: S1. Perform layered collision modeling on the arm and blood vessel models, dividing them into at least a skin layer, anterior blood vessel wall layer, blood vessel lumen layer, and posterior blood vessel wall layer, and setting independent collision boundaries for each layer, so that the model has hierarchical identifiable attributes and real-time collision response capability; S2. Real-time acquisition of the three-dimensional spatial coordinates of the virtual puncture needle tip, determination of the current tissue level of the needle tip, and generation of corresponding event trigger signals when the needle tip crosses different level boundaries. The events include "entry into the blood vessel lumen event" and "puncture of the posterior wall of the blood vessel event". The tactile-visual collaborative feedback mapping method is specifically described as follows: S3. Generate multi-layered tactile feedback signals based on the tissue layer where the needle tip is located: when the needle tip contacts the skin layer, it outputs a surface damping sensation and resistance with increasing stiffness; when the needle tip breaks through the anterior wall of the blood vessel and enters the blood vessel lumen, it outputs a distinct instantaneous breakthrough impact force and a feeling of emptiness; when the needle tip continues to advance and pierces the posterior wall of the blood vessel, the feeling of emptiness disappears, and only a low level of puncture resistance is maintained. S4. Generate corresponding visual blood return feedback signals based on the tissue layer where the needle tip is located: when it is determined that the needle tip has entered the vascular lumen layer, a red liquid column is generated inside the virtual needle tube, which rises steadily from the needle tip to the needle handle and remains there; when it is determined that the needle tip has pierced the posterior wall layer of the blood vessel, the red liquid column flashes briefly and then disappears immediately. S5. The tactile feedback signal and the visual blood return feedback signal are output synchronously based on the same event trigger signal, so that the trainee can simultaneously obtain the dual real perception of "feeling of emptiness" and "stable blood return in the needle tube" when puncturing and breaking through the blood vessel. This achieves close coordination and sensory consistency between touch and vision at key operation nodes, significantly improving the realism and operation recognition of virtual puncture training.
2. The method for tactile-visual collaborative realism simulation of intravenous puncture for virtual training of intravenous therapy nurses according to claim 1, characterized in that... The layered collision detection and event triggering method further includes a real-time high-frequency sampling step: acquiring the needle tip coordinates in real time at a sufficiently high frequency to ensure that the collision detection response has no perceptible delay, and simultaneously distributing the trigger signals of the "entry into the blood vessel lumen event" and the "puncture of the posterior wall of the blood vessel event" to the tactile feedback module and the visual blood return module.
3. The method for realistic simulation of tactile-visual co-sensory venipuncture for virtual training of intravenous therapy nurses according to claim 1, characterized in that... In the aforementioned multi-layered tactile feedback signal, the damping and resistance of the skin layer simulate the tightness of the skin when it is compressed; the instantaneous impact and sense of loss when the anterior wall of the blood vessel is broken simulate the abruptness of piercing the blood vessel wall and the physiological characteristics of the subsequent sudden drop in resistance.
4. The method for tactile-visual collaborative realism simulation of intravenous puncture for virtual training of intravenous therapy nurses according to claim 1, characterized in that... In the aforementioned visual blood return feedback signal, the steadily rising and sustained red liquid column simulates the blood return phenomenon after a successful clinical puncture, providing trainees with visual confirmation of success; the red liquid column that flashes and then disappears momentarily simulates the abnormal phenomenon of interrupted blood return after puncturing a blood vessel, forming a dual error prompt with the change in tactile pressure.
5. The method for realistic simulation of tactile-visual co-sensory venipuncture for virtual training of intravenous therapy nurses according to claim 1, characterized in that... The synchronous output of the tactile feedback signal and the visual feedback signal adopts an event-driven parallel processing architecture, which enables both to be output simultaneously based on the same event trigger signal. The output time difference is less than the human perception threshold, achieving complete consistency between touch and vision at the perception level, eliminating sensory misalignment, and enhancing immersion and realism.