A training system and method for nursing puncture and injection techniques that combines virtual and real methods

Abstract

This invention discloses a virtual-real integrated training system and method for nursing puncture and injection techniques, belonging to the field of nursing teaching and training technology. The system includes the following steps: S1, initializing and identifying a multimodal sensor network to generate training module configuration data; S2, guiding the standard procedure and detecting the preparation state before the puncture operation based on the training module configuration data to generate operation preparation state data; S3, using the built-in six-axis inertial measurement unit and linear encoder of the intelligent force feedback puncture needle to perform real-time perception and processing of the puncture needle's posture and depth, generating real-time puncture needle pose data. This invention, for the first time in the field of nursing puncture training, constructs a four-level tissue biomechanical parameterized model including the epidermal abrasion segment, subcutaneous fat segment, muscle fiber segment, and blood vessel wall breakthrough segment, and realizes real-time calculation and drive output of force-tactile feedback based on a position-force hybrid control architecture.

Description

Technical Field

[0001] This invention relates to the field of nursing teaching and training technology, and in particular to an integrated training system and method for nursing puncture and injection techniques that combines virtual and real methods. Background Technology

[0002] Nursing puncture and injection procedures are among the most frequent and challenging invasive skills in clinical nursing, involving various puncture techniques such as arterial blood collection, intramuscular injection, subcutaneous injection, intradermal injection, intravenous infusion, and indwelling intravenous catheterization. For novice nurses, the success rate of punctures highly depends on repeated practice and the accumulation of tactile experience—accurately judging the puncture depth, controlling the needle angle and depth, and recognizing the timing of blood return all require extensive training. However, existing training methods mainly rely on the following three models, each with significant technical shortcomings.

[0003] Firstly, the physical model training mode. Using static silicone arm or buttock models for puncture practice has the following technical defects: (1) It lacks the realistic breakthrough feeling of force feedback, cannot simulate the differentiated resistance characteristics of different tissue layers, and makes it difficult for operators to establish correct puncture feel memory; (2) The model has no internal sensing module, and operators cannot obtain objective data feedback on key parameters such as puncture angle, depth, and position; (3) The learning effect is highly dependent on one-on-one verbal guidance from teachers, resulting in low teaching efficiency and subjective errors in teacher evaluation. For high-risk punctures such as scalp veins in premature infants, static models cannot fully reproduce the real operating environment of extremely thin tissues and narrow blood vessels.

[0004] Secondly, the pure virtual simulation mode. Although the visualization effect is good, the three-dimensional virtual scene constructed by computer graphics is used to simulate puncture. However, there are the following technical defects: (1) It lacks the physical operation feel of real instruments. Operators cannot obtain the same mechanical feeling as in actual clinical operation, and the skill transfer effect is limited; (2) The correlation between virtual operation performance and actual clinical operation ability is unclear. Studies have shown that the needle tip position error of pure virtual training is about 40% higher on average than that of physical training; (3) It cannot simulate the multi-level differentiated force feedback in the puncture process. The training experience obtained by operators is significantly different from that of real clinical operation.

[0005] Third, the bedside practical training mode. Directly performing operation training on patients or volunteers has the following technical defects: (1) There are serious ethical risks and patient safety hazards, especially improper arterial puncture can lead to hematoma, pseudoaneurysm, and nerve damage, and improper scalp vein puncture in premature infants can lead to skull injury; (2) The standardization is extremely low, suitable patient cases are uncontrollable, and it is difficult to obtain sufficient frequency and uniform standardized repeatable training opportunities; (3) Operational errors may cause irreversible damage to patients, and the training cost and risk are extremely high.

[0006] A comprehensive analysis of existing technologies reveals the following core technological shortcomings:

[0007] (i) Lack of multi-level force-tactile feedback. Existing training models cannot accurately simulate the four levels of differentiated force feedback during puncture: the sensation of skin penetration through the epidermis, the sensation of low-resistance translation through subcutaneous fat, the sensation of resistance fluctuations through muscle fiber bundles, and the sensation of failure upon penetration through the blood vessel wall. Existing force feedback devices mostly use a simplified model with a single spring-damping mechanism, and the force refresh rate is usually below 100Hz. There is a significant delay in the force changes felt by the operator, resulting in an unrealistic feel and making it difficult for the operator to establish correct tactile memory. This is a bottleneck technical problem that restricts the effectiveness of puncture training.

[0008] (ii) Lack of real-time detection and visualization of puncture angle / depth / position. When puncturing on a physical model, the operator cannot intuitively see key information such as the direction of the needle tip in the body, angle deviation, and whether the depth exceeds the standard. Training feedback relies entirely on manual experience, resulting in a steep learning curve and a first-time clinical puncture success rate of less than 60% for novice nurses.

[0009] (iii) Lack of simulation of blood return mechanism. The sign of successful clinical puncture is "seeing blood return"—that is, blood flows back to the syringe or needle handle after the puncture needle enters the blood vessel. However, the existing training model cannot simulate the real blood return effect, and the operator cannot develop the ability to accurately judge the timing of "stopping when blood is seen, lowering the angle, and continuing to advance the tube." This is the core obstacle to the transfer of clinical puncture skills.

[0010] (iv) Multiple puncture techniques are trained separately and lack a unified platform. Different techniques such as arterial blood collection, intramuscular injection, subcutaneous injection, intradermal injection, intravenous infusion, indwelling intravenous catheter, and scalp vein puncture in premature infants usually rely on their own independent training equipment. There is a lack of a unified comprehensive training platform, resulting in high equipment procurement costs, large footprint, and complex management and maintenance.

[0011] (v) Lack of intelligent assessment adapting to age / site physiological differences. The anatomical structures of the adult arm and the head of premature infants, as well as the large muscle groups of the buttocks and the deltoid muscles of the upper arm, are vastly different. The operational technique requirements vary significantly for different age groups and different puncture sites. The existing system cannot adaptively adjust the assessment criteria and uses the same evaluation dimensions for training items of different difficulty, resulting in low assessment discrimination.

[0012] (vi) Lack of virtual-real synchronization in specific operational steps. Clinical puncture and injection involve several specific operational steps—skin tying to fix blood vessels, Allen test to assess collateral circulation, removal of the indwelling intravenous catheter core, injection of sealing solution, and observation of the wheal after intradermal injection, etc.—existing systems cannot achieve virtual-real synchronization simulation of these key steps, resulting in insufficient training completeness.

[0013] In summary, we have provided a virtual-real integrated training system and method for nursing puncture and injection techniques. Summary of the Invention

[0014] The purpose of this invention is to solve the problems in the prior art by proposing an integrated training system and method for nursing puncture and injection techniques that combines virtual and real methods.

[0015] To achieve the above objectives, the present invention adopts the following technical solution:

[0016] A training method integrating virtual and real methods for nursing puncture and injection techniques includes the following steps:

[0017] S1. Perform multimodal sensor network initialization and module identification processing to generate training module configuration data;

[0018] S2. Based on the training module configuration data, perform standard procedure guidance and preparation status detection processing before the puncture operation to generate operation preparation status data.

[0019] S3. The built-in six-axis inertial measurement unit and linear encoder of the intelligent force feedback puncture needle are used to perform real-time perception and processing of the puncture needle posture and depth, and generate real-time puncture needle posture data.

[0020] S4. Based on the real-time pose data of the puncture needle and the preset four-level tissue biomechanical parameterization model, the force feedback rendering engine performs differentiated force tactile feedback calculation and driving processing to generate a real-time force feedback driving signal.

[0021] S5. Based on the real-time pose data of the puncture needle and the preset blood vessel centerline path data, the blood return status is detected and processed by the blood vessel cavity determination algorithm to generate LED blood return indicator signal and three-dimensional blood return animation rendering data.

[0022] S6. Based on the real-time pose data of the puncture needle and the real-time force feedback drive signal, multi-mode visualization rendering processing is performed through the three-dimensional virtual-real fusion rendering engine to generate virtual-real fusion three-dimensional scene rendering data.

[0023] S7. Based on the LED blood return indicator signal, the virtual-real fusion 3D scene rendering data and the additional operation sensing data collected by the multi-module sensor network, perform virtual-real synchronization and operation timing determination processing for the entire operation process to generate operation process state sequence data.

[0024] S8. Based on the state sequence data of the operation steps, perform adaptive evaluation of operation quality through a multi-stage temporal convolutional network architecture to generate operation quality evaluation report data.

[0025] S9. Based on the operation quality assessment report data and historical training records, the training strategy is continuously optimized and personalized training schemes are generated through the intelligent analysis engine to generate personalized training scheme data.

[0026] Preferably, generating training module configuration data in S1 includes the following steps:

[0027] S11. Detect the type of the currently connected training module through the unified electrical interface of the integrated stroller. The training module includes at least one of an adult arm module, an adult hip module, and a premature infant head module.

[0028] S12. Read the electronic tag information embedded in the module to obtain the module model, dissection parameters, sensor unit calibration data and remaining service life information;

[0029] S13. Based on the read module information, initialize the system configuration, load the corresponding anatomical 3D model and force feedback parameter set, and generate the training module configuration data.

[0030] Preferably, generating operation preparation status data in step S2 includes the following steps:

[0031] S21. Based on the puncture technology corresponding to the current module type, call the preset standard operating procedure sequence and display the preparation stage operation guide in a multimodal form through the 3D virtual-real fusion rendering engine.

[0032] S22. The path and pressure distribution of the disinfection operation are detected by the force feedback touch sensor array set in each module. The sensor data is processed in real time by the edge computing node to determine whether the coverage area, number of wipes and wiping direction of the disinfection operation meet the standards.

[0033] S23. When the disinfection operation is detected to meet the preset standard, the operation preparation status data is generated.

[0034] Preferably, the method for constructing the four-level tissue biomechanical parameterization model in S4 includes:

[0035] S41. Obtain force-displacement measured data of clinical puncture operation. Collect force-time series data of the entire process of the puncture needle penetrating the epidermis, dermis, subcutaneous fat, muscle and blood vessel wall in sequence through force sensing sampling device. The sampling frequency is not less than 1000Hz.

[0036] S42. The force-time series data is segmented and labeled. According to the preset anatomical depth threshold, the force curve is divided into four stages: epidermal perforation segment, subcutaneous fat segment, muscle fiber segment, and blood vessel wall perforation segment.

[0037] S43. Perform parameter fitting on the force curve of each stage, extract the characteristic parameters of force value range, duration, mutation rate and fluctuation frequency of each stage, and construct a set of biomechanical feedback functions with depth and velocity as independent variables.

[0038] S44. The biomechanical feedback function set is associated with and stored with the corresponding anatomical layer depth threshold and velocity correction coefficient to generate the four-layer tissue biomechanical parameterization model.

[0039] Preferably, in step S4, a force feedback rendering engine is used to perform differentiated force tactile feedback calculation and driving processing, and the target feedback force corresponding to the current tissue level is calculated using the following formula:

[0040]

[0041] in, For a moment The target feedback force value, For organizational hierarchy indexing, This is the current puncture depth. For puncture speed, For case parameter vectors, For hierarchical activation functions, For the first The biomechanical feedback function of the layer;

[0042] The feedback force characteristics of the first layer of epidermis breaking through the skin are: force increment of 0.5-1.0N and force rise time of less than 50ms; the feedback force characteristics of the second layer of subcutaneous fat low-resistance translational sensation are: constant force of 0.2-0.5N; the feedback force characteristics of the third layer of muscle fiber bundle resistance fluctuation sensation are: periodic fluctuation of 0.5-1.5N with a fluctuation frequency of 10-20Hz; the feedback force characteristics of the fourth layer of blood vessel wall breaking through the skin and feeling of loss are: resistance decrease of more than 70% after the force peak of 1.0-2.0N.

[0043] The total delay of force feedback rendering is less than 5ms, the force resolution is 0.01N, and the maximum output force is 5N; the calculated target feedback force value is converted into a motor drive signal to generate the real-time force feedback drive signal.

[0044] Preferably, the generation of the LED health regeneration indicator signal and the three-dimensional health regeneration animation rendering data in step S5 includes the following steps:

[0045] S51. The three-dimensional position of the needle tip in the real-time pose data of the puncture needle is compared with the Euclidean distance of the preset blood vessel centerline path in real time through the intelligent sensor chip.

[0046] S52. When the Euclidean distance is less than a preset threshold, it is determined that the needle tip has entered the blood vessel lumen, triggering the LED lighting sequence;

[0047] S53. The arterial blood sampling needle is equipped with 4 LEDs. The first LED flashes when the needle tip touches the blood vessel wall. The second to fourth LEDs light up in sequence after the needle fails to penetrate, simulating the pulsating inflow of arterial blood. After all the LEDs are lit, they remain constant red. The venous puncture needle and indwelling needle are equipped with 1 LED, which immediately turns red after the needle fails to penetrate.

[0048] S54. The LED lighting signal synchronously drives the blood return animation rendering in the three-dimensional virtual scene. Based on the GPU Shader particle system, the fluid animation of blood entering the syringe or needle handle observation window from the blood vessel cavity is generated in real time. The number, color, and flow rate of particles are dynamically adjusted according to the hemoglobin concentration and blood pressure set in the case, generating the LED blood return indicator signal and the three-dimensional blood return animation rendering data.

[0049] Preferably, in step S7, additional operation sensing data is collected through a multi-module sensor network and processed in a virtual-real synchronization manner throughout the entire operation process. The additional operation sensing data includes: the arm rotation angle detected by a six-axis inertial attitude sensor set on the arm model; the radial artery pulsation state simulated by a micro pneumatic pulsation device set on the arm model; the operator's contact position and pressure distribution detected by a force feedback touch sensor array set on each module; the needle withdrawal operation displacement detected by a micro displacement sensor set inside the needle core of the indwelling intravenous needle; the infusion drip rate detected by an optical drop counter equipped on the infusion stand; and the skin-stretching force and direction detected by a skin-stretching force sensor.

[0050] Based on a finite state machine model, the additional operation sensor data and the real-time pose data of the puncture needle are fused and analyzed in a time sequence to determine whether the three steps of needle withdrawal are performed in the correct order, whether the skin tying action is sufficient, whether the infusion drip rate is within the doctor's order range, and whether the palm color recovery time in the Allen test is within the preset time threshold, thereby generating the state sequence data of the operation steps.

[0051] Preferably, the adaptive evaluation of operation quality in step S8 using a multi-stage temporal convolutional network architecture includes the following steps:

[0052] S81. Input the operation state sequence data into a multi-stage temporal convolutional network. In the first stage, classify the original multimodal sensing data sequence frame by frame and output the label of the operation state for each frame.

[0053] S82. The second stage performs time-series smoothing and global optimization on the output of the first stage, and outputs the time segmentation and action quality score of the complete operation.

[0054] S83. The evaluation covers five dimensions: completeness of steps, correctness of step sequence, standardization of operation, time efficiency, and safety. The evaluation results are presented in real time in a multimodal format. After training, a multi-dimensional score sheet is generated, which includes the scores of each step, error detail timestamps, comparison of standard operation, and improvement suggestions. The operation quality evaluation report data is generated.

[0055] Preferably, the process of continuously optimizing the training strategy and generating personalized training schemes through the intelligent analysis engine in S9 includes the following steps:

[0056] S91. Collect historical cycle operation quality assessment report data, time consumption data for each step, and error distribution data as historical data;

[0057] S92. Based on the historical data, the weight parameters of the evaluation model are iteratively optimized using an optimization algorithm. The parameter update is achieved through the following formula:

[0058]

[0059] in, For the first The set of parameters to be optimized in each iteration For learning rate, The gradient of the loss function with respect to the parameter set. This is a historical training dataset;

[0060] S93. Based on the optimized parameters and the identified skill weaknesses, match the preset training course library to generate targeted reinforcement training program data, and feed the optimization suggestions back to the subsequent training guidance process to generate the personalized training program data.

[0061] Preferably, the system includes a multimodal sensor network and module management module, a force feedback rendering and puncture perception module, a three-dimensional virtual-real fusion rendering module, an AI intelligent evaluation and guidance module, and an intelligent analysis and optimization module;

[0062] The multimodal sensor network and module management module automatically identify the type of training module and read the electronic tag information through a unified electrical interface. It collects data on disinfection traces, skin stretching force, arm rotation angle and needle core displacement through a multi-type sensor array, and outputs training module configuration data and additional operation sensor data.

[0063] The force feedback rendering and puncture perception module receives the configuration data of the training module, perceives the needle tip posture in real time through the six-axis inertial measurement unit and linear encoder built into the force feedback puncture needle, calculates the target feedback force through the four-level tissue biomechanical parameterization model, drives the micro linear voice coil motor to output differentiated force tactile feedback, and outputs the real-time posture data of the puncture needle and the real-time force feedback drive signal.

[0064] The three-dimensional virtual-real fusion rendering module receives the real-time pose data of the puncture needle and the real-time force feedback driving signal, performs multi-mode visualization rendering through perspective mode, cross-sectional mode and local close-up window mode, synchronously drives the three-dimensional blood return animation through the LED blood return indicator signal, and outputs virtual-real fusion three-dimensional scene rendering data.

[0065] The AI ​​intelligent assessment and guidance module receives the additional operation sensor data and the virtual-real fusion 3D scene rendering data, performs adaptive assessment of operation quality through a multi-stage temporal convolutional network architecture, supports natural language interaction and virtual patient scenario dialogue through a speech recognition engine, and outputs operation quality assessment report data and guidance instruction data.

[0066] The intelligent analysis and optimization module receives the operation quality assessment report data and historical training records, evaluates the historical training effect through the scheduling efficiency analysis unit, generates evaluation model parameter adjustment suggestions through the strategy parameter optimization unit, matches the training course library through the personalized scheme generation unit, and outputs personalized training scheme data to the AI ​​intelligent assessment and guidance module.

[0067] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0068] 1. This invention is the first in the field of nursing puncture training to construct a four-level tissue biomechanical parameterized model including the epidermal abrasion segment, subcutaneous fat segment, muscle fiber segment, and blood vessel wall breakthrough segment. Based on the position-force hybrid control architecture, it realizes real-time calculation and drive output of force tactile feedback, effectively solving the technical defects of the single spring-damping model in the prior art, such as unrealistic feel, low force refresh rate, and large delay. This enables operators to establish correct puncture feel memory and significantly shortens the clinical learning curve.

[0069] 2. This invention integrates a six-axis inertial measurement unit, a linear encoder, and multiple LED beads into a force feedback puncture needle. When the needle tip enters the blood vessel lumen, the four LEDs of the arterial blood sampling puncture needle light up sequentially at intervals to simulate pulsatile blood return, while one LED of the venous puncture needle immediately lights up red. Simultaneously, the GPU Shader particle system in the three-dimensional virtual scene is driven to generate a blood flow animation, realizing a three-modal information synchronous closed loop of "force feedback feeling of failure → LED sequence visual confirmation → three-dimensional software blood return animation". Based on this, the operator can establish a precise timing judgment ability of "stopping upon seeing blood, lowering the angle, and continuing to advance the tube", solving the core technical problems of the complete lack of blood return mechanism simulation in the prior art and the inability of the operator to obtain objective puncture success signals.

[0070] 3. This invention utilizes a finite state machine model to perform temporal fusion analysis of multi-source sensor data. Through a three-step state machine for needle core removal, it achieves simultaneous virtual and real coverage of all key operational steps in the entire process—from Allen test → disinfection → skin bandaging → puncture → blood return → catheter insertion → needle core removal → fixation → drip rate adjustment → catheter sealing → needle removal → compression hemostasis—without any training blind spots. Simultaneously, a multi-stage temporal convolutional network can perform frame-by-frame action classification and temporal smoothing optimization of multimodal sensor data sequences, providing objective quantitative evaluation across five dimensions: step completeness, step sequence correctness, operational standardization, time efficiency, and safety. The evaluation consistency rate is significantly better than traditional subjective human evaluation. Combined with cloud-based learning archives and big data analysis capabilities, it can accurately pinpoint the operator's skill weaknesses and automatically generate personalized reinforcement training plans, achieving continuous self-optimization of the training strategy. This effectively solves the technical problems of fragmented training ("one skill, one machine"), inconsistent evaluation standards, and low teaching efficiency in existing technologies. Attached Figure Description

[0071] Figure 1 This is a flowchart of an integrated training method combining virtual and real methods for nursing puncture and injection techniques according to the present invention;

[0072] Figure 2 This is a diagram illustrating the architecture of a virtual-real integrated training system for nursing puncture and injection techniques according to the present invention. Detailed Implementation

[0073] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0074] Please see Figure 1 - Figure 2 The present invention provides a training method that combines virtual and real methods for nursing puncture and injection techniques, which specifically includes the following steps.

[0075] Step 1: Multimodal sensor network initialization and module identification processing

[0076] This step is fundamental to the entire method, aiming to automatically identify and standardize the configuration of diverse training modules at the underlying level. The system automatically detects the type of currently installed training module through the unified electrical interface of the integrated cart, reading the electronic tag information embedded in the module to obtain the module model, anatomical parameters, sensor unit calibration data, and remaining service life information. Module identification is achieved through the following identification decision function:

[0077]

[0078] in, For module type decision function, This refers to the electronic tag signal vector read from the electrical interface. For module type index, For tag-based identity recognition functions, To calibrate the data verification function, and These are the identity weight and verification weight for the corresponding module type. Based on the recognition results, the system loads the corresponding anatomical 3D model and force feedback parameter set to generate training module configuration data.

[0079] Step 2: Standard procedure guidance and preparation status monitoring before puncture operation

[0080] After obtaining the module configuration, the system, based on the puncture technology corresponding to the current module type, invokes a preset standard operating procedure sequence and displays the preparation phase operation guidance in a multimodal format using a 3D virtual-real fusion rendering engine. Disinfection detection is achieved through a force feedback touch sensor array, and disinfection trace detection is achieved using the following formula:

[0081]

[0082] in, Indicates position Confidence level of disinfection coverage at the location. For the first Coordinates of each sensor sampling point The contact radius of the cotton swab. This is the pressure weighting factor. This represents the total number of sampling points. Once the coverage area, number of wipes, and wiping direction of the disinfection operation meet the preset standards, operation preparation status data is generated.

[0083] Step 3: Real-time sensing and processing of puncture needle posture and depth

[0084] This step is the core of the entire system's sensing capabilities, designed to acquire high-precision real-time data on the spatial state of the puncture needle during the procedure. The force feedback puncture needle incorporates a six-axis inertial measurement unit (IMU) and a linear encoder. The IMU sampling frequency is no less than 200Hz, with angle measurement accuracy better than ±1° and depth measurement accuracy better than ±0.5mm. Attitude calculation is achieved using the following quaternion update formula:

[0085]

[0086] in, For the first The attitude quaternion at each sampling time. The three-axis angular velocity vector output by the IMU. The sampling interval is... This is a quaternion multiplication operator. Combined with the displacement data output from the linear encoder, it generates real-time pose data for the puncture needle, including its three-dimensional position and three-dimensional orientation.

[0087] Step 4: Calculation and Driving Process of Four-Level Differentiated Force-Haptic Feedback

[0088] This step is the core of the entire system's force feedback, designed to generate a differentiated feedback force based on the tissue layer where the needle tip is located. The force feedback rendering engine, based on a position-force hybrid control architecture, calculates the target feedback force corresponding to the current tissue layer using the following formula:

[0089]

[0090] in, For a moment The target feedback force value, For organizational hierarchy indexing, This is the current puncture depth. For puncture speed, For case parameter vectors, For hierarchical activation functions, For the first The biomechanical feedback function for each layer is set as follows: For epidermal skin penetration, the force increment is 0.5-1.0 N with a rise time of less than 50 ms; for subcutaneous fat, the force is constant at 0.2-0.5 N for low-resistance translation; for muscle fiber bundle resistance fluctuations, the force fluctuates periodically at 0.5-1.5 N with a frequency of 10-20 Hz; and for blood vessel wall penetration, the force peak is 1.0-2.0 N followed by a resistance reduction of over 70%. The total force feedback rendering delay is less than 5 ms, the force resolution is 0.01 N, and the maximum output force is 5 N.

[0091] Step 5: Vascular cavity identification and multi-level LED blood return indication processing

[0092] This step achieves accurate simulation of the blood return mechanism and is crucial for transferring clinical puncture skills. The system uses an intelligent sensor chip to compare the Euclidean distance between the three-dimensional position of the needle tip and the preset blood vessel centerline path in real time. When the distance is less than a preset threshold, it determines that the needle tip has entered the blood vessel lumen, triggering an LED lighting sequence. Blood vessel lumen determination is achieved through the following piecewise function:

[0093]

[0094] in, The three-dimensional position of the needle tip. Location of the blood vessel wall. This is the location of the blood vessel's centerline. The radius for determining the blood vessel wall. The radius of the lumen is used for determination. The arterial blood sampling needle is equipped with 4 LEDs. The first LED flashes when the needle tip touches the blood vessel wall. The second to fourth LEDs light up sequentially at intervals of about 50ms after the needle tip misses the vessel wall, simulating the pulsating influx of arterial blood. The venous puncture needle is equipped with 1 LED, which lights up red immediately after the needle tip misses the vessel wall. The LED lighting signals synchronously drive the rendering of a 3D blood return animation.

[0095] Step Six: 3D Virtual-Real Fusion Multi-Mode Visualization Rendering Process

[0096] This step achieves "clear visibility" of the puncture process, meeting the needs of different training stages through three visualization modes: perspective mode, cross-sectional mode, and close-up window mode. In perspective mode, the 3D model's skin and superficial tissues are semi-transparent, and a screen-space ambient occlusion shader generates dark shadows at the contact edges between blood vessels and bones to enhance depth perception. In cross-sectional mode, a ClippingPlaneShader performs real-time GPU cross-sectioning, exposing the 2D cross-section or semi-cross-sectional 3D morphology of the target tissue, supporting dual-window display. In close-up window mode, a PiP window automatically pops up at critical moments of the puncture, magnifying the 3D spatial relationship between the needle tip and the target tissue by 2-5 times. The 3D scene rendering frame rate is no less than 60fps.

[0097] Step 7: Simultaneous virtual and real-world synchronization and operation timing determination throughout the entire operation process.

[0098] This step achieves simultaneous virtual and real coverage of the entire process from Allen test to pressure hemostasis. Based on a finite state machine model, time-series fusion analysis of multi-source sensor data is performed, and the three-step needle withdrawal determination is implemented through the following finite state machine definition:

[0099]

[0100]

[0101] in, This is a limited state for needle core removal operation. This is the initial state. The needle core has been slightly removed (remove the needle core about 0.5cm first). To ensure the needle core is retracted while the outer tube is being inserted, The needle core is fully withdrawn. This is an error state; if any state transition sequence is incorrect or the Δd deviation exceeds a preset threshold, then it enters an error state. The status is displayed and points are deducted, along with an error message.

[0102] Allen's experiment achieved real-time synchronization of virtual and real effects through GPU shader rendering of skin color gradients combined with automatic countdown judgment; real-time synchronization of skin stretching animation was achieved through force sensors detecting skin stretching tension and driving real-time deformation of GPU skeletal skinning animation, with the deformation amount driven by the following formula:

[0103]

[0104] in, This represents the displacement of the skin apex. The skin elasticity coefficient, This refers to the tensile strength value for skin tightening. The tensile force direction vector is used, and the maximum deformation is controlled within the physiological range of skin stretching rate not exceeding 30%. Intradermal injection wheal simulation is based on a mass-spring model to calculate liquid diffusion and skin elevation in real time. The 3D rendering engine automatically measures the wheal diameter and compares it with a standard range.

[0105] Step 8: Adaptive evaluation of operation quality based on multi-stage temporal convolutional networks

[0106] This step enables real-time, objective, and quantitative evaluation of the puncture operation. The multi-stage temporal convolutional network performs frame-by-frame action classification on the original multimodal sensor data sequence in the first stage, outputting a label for the operation state of each frame. The second stage performs temporal smoothing and global optimization on the output of the first stage, outputting the time segmentation and action quality score of the complete operation. The evaluation covers five dimensions: step completeness (30%), step sequence correctness (25%), operation standardization (25%), time efficiency (10%), and safety (10%). The evaluation results are presented in real-time multimodal form, and a multi-dimensional score report is generated after training, including scores for each step, error detail timestamps, comparisons with standard operations, and improvement suggestions.

[0107] Step Nine: Continuous Optimization of Training Strategies and Generation of Personalized Training Plans

[0108] This step enables the system to learn and continuously optimize itself. The system collects historical operational quality assessment data and iteratively optimizes the assessment model parameters using an optimization algorithm. Parameter updates are achieved through the following formula:

[0109]

[0110] in, For the first The set of parameters to be optimized in each iteration (including the evaluation model weight matrix, bias vector, and tidal scheduling strategy parameters). For learning rate, The gradient of the loss function with respect to the parameter set. The dataset used is the historical training dataset. The overall loss function is a weighted average of the matching loss and the policy loss. Based on the optimized parameters and the identified skill weaknesses, a personalized reinforcement training plan is generated by matching the pre-set training course library, forming a complete closed loop of perception, decision-making, execution, evaluation, and optimization.

[0111] Step 10: Overall System Architecture

[0112] Please see Figure 2 The present invention provides a virtual-real integrated training system for nursing puncture and injection technology, which includes a multimodal sensor network and module management module, a force feedback rendering and puncture perception module, a three-dimensional virtual-real fusion rendering module, an AI intelligent assessment and guidance module, and an intelligent analysis and optimization module.

[0113] The multimodal sensor network and module management module automatically identify the training module type and read the electronic tag information through a unified electrical interface. It collects data on disinfection traces, skin stretching force, arm rotation angle, and needle core displacement through a multi-type sensor array, and outputs training module configuration data and additional operation sensor data.

[0114] The force feedback rendering and puncture perception module receives the configuration data of the training module, perceives the needle tip pose in real time through the six-axis inertial measurement unit and linear encoder built into the force feedback puncture needle, calculates the target feedback force through a four-level tissue biomechanical parameterization model, and drives a micro linear voice coil motor to output differentiated force tactile feedback, outputting real-time puncture needle pose data and real-time force feedback drive signal.

[0115] The three-dimensional virtual-real fusion rendering module receives the real-time pose data of the puncture needle and the real-time force feedback driving signal, performs multi-mode visualization rendering through perspective mode, cross-sectional mode and local close-up window mode, synchronously drives the three-dimensional blood return animation through the LED blood return indicator signal, and outputs virtual-real fusion three-dimensional scene rendering data.

[0116] The AI ​​intelligent assessment and guidance module receives the additional operation sensing data and the virtual-real fusion 3D scene rendering data, performs adaptive assessment of operation quality through a multi-stage temporal convolutional network architecture, supports natural language interaction and virtual patient scenario dialogue through a speech recognition engine, and outputs operation quality assessment report data and guidance instruction data.

[0117] The intelligent analysis and optimization module receives the operation quality assessment report data and historical training records, evaluates the historical training effect through the scheduling efficiency analysis unit, generates evaluation model parameter adjustment suggestions through the strategy parameter optimization unit, matches the training course library through the personalized scheme generation unit, and outputs personalized training scheme data to the AI ​​intelligent assessment and guidance module.

[0118] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A training method integrating virtual and real methods for nursing puncture and injection techniques, characterized in that, Includes the following steps: S1. Perform multimodal sensor network initialization and module identification processing to generate training module configuration data; S2. Based on the training module configuration data, perform standard procedure guidance and preparation status detection processing before the puncture operation to generate operation preparation status data. S3. The built-in six-axis inertial measurement unit and linear encoder of the intelligent force feedback puncture needle are used to perform real-time perception and processing of the puncture needle posture and depth, and generate real-time puncture needle posture data. S4. Based on the real-time pose data of the puncture needle and the preset four-level tissue biomechanical parameterization model, the force feedback rendering engine performs differentiated force tactile feedback calculation and driving processing to generate real-time force feedback driving signals. S5. Based on the real-time pose data of the puncture needle and the preset blood vessel centerline path data, the blood return status is detected and processed by the blood vessel cavity determination algorithm to generate LED blood return indicator signal and three-dimensional blood return animation rendering data. S6. Based on the real-time pose data of the puncture needle and the real-time force feedback drive signal, multi-mode visualization rendering processing is performed through the three-dimensional virtual-real fusion rendering engine to generate virtual-real fusion three-dimensional scene rendering data. S7. Based on the LED blood return indicator signal, the virtual-real fusion 3D scene rendering data and the additional operation sensor data collected by the multi-module sensor network, perform virtual-real synchronization and operation timing determination processing for the entire operation process to generate operation process state sequence data. S8. Based on the state sequence data of the operation steps, perform adaptive evaluation of operation quality through a multi-stage temporal convolutional network architecture to generate operation quality evaluation report data. S9. Based on the operation quality assessment report data and historical training records, the training strategy is continuously optimized and personalized training schemes are generated through the intelligent analysis engine to generate personalized training scheme data.

2. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... The process of generating training module configuration data in S1 includes the following steps: S11. Detect the type of the currently connected training module through the unified electrical interface of the integrated stroller. The training module includes at least one of an adult arm module, an adult hip module, and a premature infant head module. S12. Read the electronic tag information embedded in the module to obtain the module model, dissection parameters, sensor unit calibration data and remaining service life information; S13. Based on the read module information, initialize the system configuration, load the corresponding anatomical 3D model and force feedback parameter set, and generate the training module configuration data.

3. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... The steps involved in generating the operation preparation status data in step S2 are as follows: S21. Based on the puncture technology corresponding to the current module type, call the preset standard operating procedure sequence and display the preparation stage operation guide in a multimodal form through the 3D virtual-real fusion rendering engine. S22. The path and pressure distribution of the disinfection operation are detected by the force feedback touch sensor array set in each module. The sensor data is processed in real time by the edge computing node to determine whether the coverage area, number of wipes and wiping direction of the disinfection operation meet the standards. S23. When the disinfection operation is detected to meet the preset standard, the operation preparation status data is generated.

4. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... The method for constructing the four-level tissue biomechanical parameterization model in S4 includes: S41. Obtain force-displacement measured data of clinical puncture operation. Collect force-time series data of the entire process of the puncture needle penetrating the epidermis, dermis, subcutaneous fat, muscle and blood vessel wall in sequence through force sensing sampling device. The sampling frequency is not less than 1000Hz. S42. The force-time series data is segmented and labeled. According to the preset anatomical depth threshold, the force curve is divided into four stages: epidermal perforation segment, subcutaneous fat segment, muscle fiber segment, and blood vessel wall perforation segment. S43. Perform parameter fitting on the force curve of each stage, extract the characteristic parameters of force value range, duration, mutation rate and fluctuation frequency of each stage, and construct a set of biomechanical feedback functions with depth and velocity as independent variables. S44. The biomechanical feedback function set is associated with and stored with the corresponding anatomical layer depth threshold and velocity correction coefficient to generate the four-layer tissue biomechanical parameterization model.

5. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... In S4, a force feedback rendering engine is used to perform differentiated force tactile feedback calculation and driving processing. The target feedback force corresponding to the current tissue level is calculated using the following formula: in, For a moment The target feedback force value, For organizational hierarchy indexing, This is the current puncture depth. For puncture speed, For case parameter vectors, For hierarchical activation functions, For the first Biomechanical feedback function of the layer; The feedback force characteristics of the first layer of epidermis breaking through the skin are: force increment of 0.5-1.0N and force rise time of less than 50ms; the feedback force characteristics of the second layer of subcutaneous fat low-resistance translational sensation are: constant force of 0.2-0.5N; the feedback force characteristics of the third layer of muscle fiber bundle resistance fluctuation sensation are: periodic fluctuation of 0.5-1.5N with a fluctuation frequency of 10-20Hz; the feedback force characteristics of the fourth layer of blood vessel wall breaking through the skin and feeling of loss are: resistance decrease of more than 70% after the force peak of 1.0-2.0N. The total delay of force feedback rendering is less than 5ms, the force resolution is 0.01N, and the maximum output force is 5N; the calculated target feedback force value is converted into a motor drive signal to generate the real-time force feedback drive signal.

6. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... The steps involved in generating the LED health regeneration indicator signal and the 3D health regeneration animation rendering data in step S5 are as follows: S51. The three-dimensional position of the needle tip in the real-time pose data of the puncture needle is compared with the Euclidean distance of the preset blood vessel centerline path in real time through the intelligent sensor chip. S52. When the Euclidean distance is less than a preset threshold, it is determined that the needle tip has entered the blood vessel lumen, triggering the LED lighting sequence; S53. The arterial blood sampling needle is equipped with 4 LEDs. The first LED flashes when the needle tip touches the blood vessel wall. The second to fourth LEDs light up in sequence after the needle fails to penetrate, simulating the pulsating inflow of arterial blood. After all the LEDs are lit, they remain constant red. The venous puncture needle and indwelling needle are equipped with 1 LED, which immediately turns red after the needle fails to penetrate. S54. The LED lighting signal synchronously drives the blood return animation rendering in the three-dimensional virtual scene. Based on the GPU Shader particle system, the fluid animation of blood entering the syringe or needle handle observation window from the blood vessel cavity is generated in real time. The number, color, and flow rate of particles are dynamically adjusted according to the hemoglobin concentration and blood pressure set in the case, generating the LED blood return indicator signal and the three-dimensional blood return animation rendering data.

7. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... In S7, additional operation sensing data is collected through a multi-module sensor network and processed in a virtual-real synchronization manner throughout the entire operation process. The additional operation sensing data includes: the arm rotation angle detected by a six-axis inertial attitude sensor set in the arm model; the radial artery pulsation state simulated by a micro pneumatic pulsation device set in the arm model; the operator's contact position and pressure distribution detected by a force feedback touch sensor array set in each module; the needle withdrawal operation displacement detected by a micro displacement sensor set inside the needle core of the indwelling intravenous needle; the infusion drip rate detected by an optical drop counter equipped on the infusion stand; and the skin-stretching tension and direction detected by a skin-stretching force sensor. Based on a finite state machine model, the additional operation sensor data and the real-time pose data of the puncture needle are fused and analyzed in a time sequence to determine whether the three steps of needle withdrawal are performed in the correct order, whether the skin tying action is sufficient, whether the infusion drip rate is within the doctor's order range, and whether the palm color recovery time in the Allen test is within the preset time threshold, thereby generating the state sequence data of the operation steps.

8. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... The adaptive evaluation of operational quality in S8 using a multi-stage temporal convolutional network architecture includes the following steps: S81. Input the operation state sequence data into a multi-stage temporal convolutional network. In the first stage, classify the original multimodal sensing data sequence frame by frame and output the label of the operation state for each frame. S82. The second stage performs time-series smoothing and global optimization on the output of the first stage, and outputs the time segmentation and action quality score of the complete operation. S83. The evaluation covers five dimensions: completeness of steps, correctness of step sequence, standardization of operation, time efficiency, and safety. The evaluation results are presented in real time in a multimodal format. After training, a multi-dimensional score sheet is generated, which includes the scores of each step, error detail timestamps, comparison of standard operation, and improvement suggestions. The operation quality evaluation report data is generated.

9. The integrated training method for nursing puncture and injection techniques, combining virtual and real methods, as described in claim 1, is characterized in that... The S9 process, which uses an intelligent analysis engine to continuously optimize training strategies and generate personalized training schemes, includes the following steps: S91. Collect historical cycle operation quality assessment report data, time consumption data for each step, and error distribution data as historical data; S92. Based on the historical data, the weight parameters of the evaluation model are iteratively optimized using an optimization algorithm. The parameter update is achieved through the following formula: in, For the first The set of parameters to be optimized in each iteration For learning rate, The gradient of the loss function with respect to the parameter set. This is a historical training dataset; S93. Based on the optimized parameters and the identified skill weaknesses, match the preset training course library to generate targeted reinforcement training program data, and feed the optimization suggestions back to the subsequent training guidance process to generate the personalized training program data.

10. A virtual-real integrated training system for nursing puncture and injection techniques, characterized in that, The system is used to implement the integrated training method for virtual and real combination of nursing puncture and injection technology as described in any one of claims 1-9. The system includes a multimodal sensor network and module management module, a force feedback rendering and puncture perception module, a three-dimensional virtual and real fusion rendering module, an AI intelligent assessment and guidance module, and an intelligent analysis and optimization module. The multimodal sensor network and module management module automatically identify the type of training module and read the electronic tag information through a unified electrical interface. It collects data on disinfection traces, skin stretching force, arm rotation angle and needle core displacement through a multi-type sensor array, and outputs training module configuration data and additional operation sensor data. The force feedback rendering and puncture perception module receives the configuration data of the training module, perceives the needle tip posture in real time through the six-axis inertial measurement unit and linear encoder built into the force feedback puncture needle, calculates the target feedback force through the four-level tissue biomechanical parameterization model, drives the micro linear voice coil motor to output differentiated force tactile feedback, and outputs the real-time posture data of the puncture needle and the real-time force feedback drive signal. The three-dimensional virtual-real fusion rendering module receives the real-time pose data of the puncture needle and the real-time force feedback driving signal, performs multi-mode visualization rendering through perspective mode, cross-sectional mode and local close-up window mode, synchronously drives the three-dimensional blood return animation through the LED blood return indicator signal, and outputs virtual-real fusion three-dimensional scene rendering data. The AI ​​intelligent assessment and guidance module receives the additional operation sensor data and the virtual-real fusion 3D scene rendering data, performs adaptive assessment of operation quality through a multi-stage temporal convolutional network architecture, supports natural language interaction and virtual patient scenario dialogue through a speech recognition engine, and outputs operation quality assessment report data and guidance instruction data. The intelligent analysis and optimization module receives the operation quality assessment report data and historical training records, evaluates the historical training effect through the scheduling efficiency analysis unit, generates evaluation model parameter adjustment suggestions through the strategy parameter optimization unit, matches the training course library through the personalized scheme generation unit, and outputs personalized training scheme data to the AI ​​intelligent assessment and guidance module.

Images