Puncture teaching simulation system and method based on real-time quantification of operation trajectory

Abstract

This invention relates to the field of intelligent puncture teaching technology, and in particular to a puncture teaching simulation system and method based on real-time quantification of operation trajectory. The system includes: reconstructing a three-dimensional virtual puncture environment based on patient medical images to provide trainees with a highly realistic simulated operation scenario, enhancing the immersion and realism of the teaching; comprehensively recording trainees' operational details by real-time acquisition of multi-dimensional spatiotemporal operation data streams, providing data support for subsequent quantitative analysis and evaluation; generating skill profiles by extracting and fusing quantitative features from multiple dimensions to achieve a comprehensive, objective, and quantitative evaluation of trainees' operational skills, accurately reflecting their skill level; generating evaluation results through comparative analysis to help trainees understand their shortcomings and improve accordingly; and dynamically adjusting the virtual environment based on trainees' skill profiles and evaluation results to generate personalized training scenario sequences, meeting the training needs of trainees at different skill stages and improving teaching effectiveness.

Description

Technical Field

[0001] This invention relates to the field of intelligent puncture teaching technology, and in particular to a puncture teaching simulation system and method based on real-time quantification of operation trajectory. Background Technology

[0002] With the popularization of minimally invasive interventional surgery, percutaneous puncture has become a routine clinical procedure. The accuracy of this procedure is directly related to the treatment effect and patient safety. The existing puncture skills training system mainly relies on the traditional model: trainees gain a feel for the procedure by practicing on physical models and learn decision-making by watching the instructor's surgical videos. Some virtual reality simulation systems provide trainees with a risk-free repetitive training environment through three-dimensional visualization and simple force feedback, demonstrating value in improving spatial cognition and hand-eye coordination.

[0003] However, puncture is an operation that demands a high degree of multi-dimensional skill coordination, requiring not only precise spatial positioning and keen biomechanical awareness, but also dynamic synchronization with the patient's respiratory cycle. Existing training methods face fundamental limitations in this regard: physical models cannot simulate real physiological movements, and training results heavily rely on the instructor's subjective and experiential evaluation, lacking objective quantification of the intrinsic quality of the puncture trajectory; while existing virtual reality simulation systems mostly only reproduce visual scenes and basic force perception; these systems cannot deeply analyze the multi-dimensional skill characteristics of the operation process, thus failing to identify the trainee's specific weaknesses in key dimensions, and even less to generate personalized training paths. This results in skill training remaining at a crude imitation stage, making it difficult to achieve the leap from completing the operation to mastering it, and failing to meet the urgent clinical need for training high-level, standardized interventional physicians. To address this, this invention proposes a puncture teaching simulation system and method based on real-time quantification of the puncture trajectory. Summary of the Invention

[0004] The purpose of this invention is to solve the problems in the background art, and to propose a puncture teaching simulation system and method based on real-time quantification of operation trajectory.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A puncture teaching simulation system based on real-time quantification of the operation trajectory includes:

[0007] The virtual teaching scenario construction module is used to reconstruct a three-dimensional virtual puncture environment based on patient sequence medical images;

[0008] The multimodal interaction and operation acquisition module is used to acquire multidimensional spatiotemporal operation data streams generated by trainees when performing puncture simulation in a three-dimensional virtual puncture environment in real time.

[0009] The operational skill quantification analysis module is used to preprocess multi-dimensional spatiotemporal operational data streams, extract and fuse quantitative features of spatial accuracy, tissue biomechanics identification, physiological motor coordination and operational fluency dimensions, and generate skill profiles.

[0010] The intelligent assessment and adaptive feedback module is used to compare and analyze the trainee's skill profile with pre-stored multi-category standard operating modes to generate assessment results that include deviation information in various dimensions.

[0011] The teaching path dynamic planning module is used to dynamically adjust the subsequent 3D virtual puncture environment based on the student's skill profile and assessment results, thereby generating a personalized training scenario sequence that is adapted to the student's current skill level.

[0012] Furthermore, the three-dimensional virtual puncture environment is reconstructed, including:

[0013] The image analysis and spatial mapping unit is used to import sequential medical images containing puncture entry point A and target point B. It extracts the boundaries of different tissues through a segmentation algorithm based on a multi-scale feature pyramid network, and maps the two-dimensional pixel coordinates to a unified world coordinate system according to the physical resolution of the images and the inter-layer positional relationship, thus constructing a voxelized three-dimensional tissue model.

[0014] The physiological dynamic simulation unit is used to bind preset dynamic equations to different anatomical structures in a three-dimensional tissue model to simulate tissue deformation and displacement caused by physiological movement, and output real-time physiological state signals, which include at least the respiratory phase.

[0015] The risk field mapping unit is used to generate a spatially continuous risk attenuation field in the space of a three-dimensional organizational model, using the surface of the extracted hazardous structure as the source.

[0016] Furthermore, the system collects multi-dimensional spatiotemporal operational data streams in real time during puncture simulations in a 3D virtual puncture environment, including:

[0017] The spatial coordinates and posture data of the end of the force feedback puncture instrument are collected by an optical positioning device to form a time-continuous puncture trajectory sequence.

[0018] Simultaneously record six-dimensional force / torque data generated by the interaction between the force feedback puncture instrument simulation and a physical simulation entity model, forming a mechanical interaction sequence;

[0019] Detect and record key state transition events during the virtual puncture process, including fascia penetration events, target area entry events, and high-risk invasion events in the risk field, forming a state transition event sequence;

[0020] The puncture trajectory sequence, mechanical interaction sequence, and state transition event sequence are synchronized and time-aligned to form a multidimensional spatiotemporal operational data stream.

[0021] Furthermore, the specific method for detecting and recording high-risk intrusion events in the risk field is as follows: real-time calculation of the risk value of the puncture instrument tip in the risk decay field. When the risk value is continuously higher than the preset risk alarm threshold and reaches the preset duration, a high-risk intrusion event is determined to be triggered, and the corresponding area is recorded as the high-value area.

[0022] Furthermore, quantitative features are extracted and fused to generate skill profiles, including:

[0023] Based on the puncture trajectory sequence, the dynamic time warp distance between it and the planned path is calculated as the path deviation, and the cumulative time proportion of trajectory points falling into the high-value area in the risk decay field is counted as the risk assessment index, which together constitute the spatial accuracy characteristics.

[0024] Based on the mechanical interaction sequence and state transition event sequence, force curve segments associated with fascia penetration events and target area entry events are identified. The mean of the force rise slope and the fluctuation variance of the force curve segments are calculated as tissue mechanical identification features.

[0025] Based on the instantaneous velocity sequence extracted from the puncture trajectory sequence and the real-time respiratory phase signal output by the physiological dynamic simulation unit, the percentage of displacement occurring within a preset stable phase window in the respiratory cycle is calculated as the physiological motion coordination feature.

[0026] Simultaneously, the proportion of effective needle insertion time to total operation time is calculated, and the sample entropy of the needle insertion speed sequence is calculated as a feature of operation smoothness.

[0027] All calculated features are normalized and weighted and fused to output a multi-dimensional skill feature vector, which represents the skill profile of a single operation.

[0028] Furthermore, the method for calculating the mean of the force rise slope and the variance of the fluctuation includes: after smoothing and filtering the force curve segment, obtaining its first time derivative, taking the arithmetic mean of the derivative sequence within a preset time window before and after the event occurrence time as the mean of the force rise slope, and calculating the standard deviation of the derivative sequence within the window as the variance of the force fluctuation.

[0029] Furthermore, the trainee's skill profile is compared and analyzed with pre-stored multi-category standard operating procedures to generate an evaluation result containing deviation information across various dimensions, including:

[0030] Receive skill profiles; input the skill feature vectors generated by the student's operations into a knowledge base that has multiple sets of feature vectors for standard operation modes stored in advance;

[0031] Calculate the Euclidean distance between the skill feature vector of the trainee's operation and the center of the set of feature vectors of each standard operation mode in the knowledge base in the feature space composed of the dimensions of spatial precision, organizational mechanical identification, physiological motor coordination and operation fluency.

[0032] The trainee's operation mode category is determined based on the minimum distance principle. The evaluation deviation value of each dimension of the trainee's skill feature vector is obtained by subtracting the value of each dimension of the central feature vector of the corresponding standard operation mode from the value of each dimension of the skill feature vector.

[0033] Based on the evaluation deviation values ​​of each dimension, a dynamic feedback strategy is generated, which triggers targeted visual, force, or auditory feedback instructions.

[0034] Furthermore, the subsequent 3D virtual puncture environment is dynamically adjusted to generate a personalized training scenario sequence adapted to the trainee's current skill level, including:

[0035] Based on the trainees' historical skill profiles and their evaluation deviations in various dimensions such as spatial accuracy, organizational biomechanics identification, physiological-motor coordination, and operational fluency, we analyze the trends in skill development.

[0036] Based on the analysis of skill development trends, a personalized training scenario sequence is generated, consisting of multiple three-dimensional virtual puncture environments with different scenario parameter configurations. Each training scenario in the sequence is matched or challenged to the trainee's current skill level by adjusting at least one scenario parameter, including anatomical complexity, physiological and motor interference intensity, and risk field distribution.

[0037] If a trainee's assessment deviation value in any dimension exceeds the preset threshold for that dimension, and the skill development trend analysis shows that the dimension represents a skill bottleneck, then in subsequent personalized training scenario sequences, the challenge level of the scenario parameters corresponding to that dimension will be increased. If a trainee's assessment deviation value in all dimensions is lower than the preset threshold for the corresponding dimension, and the skill development trend analysis shows that the performance in each dimension is stable or improving, then a comprehensive high-level challenge scenario will be generated and arranged as a subsequent component of the personalized training scenario sequence.

[0038] Furthermore, during the movement of the robotic arm, the breathing phase is continuously monitored. If the breathing phase exits the safe phase range prematurely during the authorized execution period, the authorization is immediately revoked and the movement of the robotic arm is paused, waiting for the next safe phase range to arrive before being retried.

[0039] A puncture teaching simulation method based on real-time quantification of the operation trajectory includes:

[0040] S1. Reconstruct a three-dimensional virtual puncture environment based on the patient's sequence medical images;

[0041] S2. Real-time acquisition of multi-dimensional spatiotemporal operation data streams generated when trainees perform puncture simulations in a three-dimensional virtual puncture environment;

[0042] S3. Preprocess the multidimensional spatiotemporal operation data stream, extract and fuse quantitative features of spatial accuracy, tissue mechanics identification, physiological motion coordination and operation fluency dimensions to generate a skill profile;

[0043] S4. Compare and analyze the trainee's skill profile with the pre-stored multi-category standard operating modes to generate an evaluation result that includes deviation information in each dimension.

[0044] S5. Based on the trainee's skill profile and assessment results, dynamically adjust the subsequent three-dimensional virtual puncture environment to generate a personalized training scenario sequence that suits the trainee's current skill level.

[0045] Compared with existing technologies, the beneficial effects of this invention are as follows: By reconstructing a three-dimensional virtual puncture environment based on patient sequence medical images, it provides trainees with a highly realistic simulated operation scenario. The image analysis and spatial mapping unit constructs a voxelized three-dimensional tissue model, the physiological dynamic simulation unit simulates tissue changes caused by physiological movement, and the risk field mapping unit generates a risk attenuation field, allowing trainees to train in a near-realistic environment, enhancing learning immersion and practicality. Furthermore, by real-time acquisition of multi-dimensional spatiotemporal operation data streams, it comprehensively records the trainee's operational details. Spatial coordinates and posture data are acquired through an optical positioning device, synchronously recording mechanical interactions and key state transition events, and performing time alignment, providing a rich and accurate data foundation for subsequent analysis. To ensure comprehensive and objective assessment, the system preprocesses operational data, extracts and integrates multi-dimensional quantitative features to generate skill profiles. It quantitatively assesses trainees' skills in areas such as spatial accuracy, organizational biomechanics identification, physiological-motor coordination, and operational fluency, making the assessment results more scientific and facilitating the identification of trainees' strengths and weaknesses. By comparing and analyzing the skill profiles with standard models, the system generates assessment results and triggers targeted feedback instructions, helping trainees understand the gap between their operations and the standards, guiding them to improve their operations and enhance learning outcomes. Furthermore, by dynamically adjusting the subsequent environment based on the skill profiles and assessment results, the system generates personalized training scenario sequences that adapt to trainees' skill levels, gradually increasing the challenge difficulty and promoting continuous skill improvement, thus achieving individualized instruction. Attached Figure Description

[0046] Figure 1 This is a block diagram of a puncture teaching simulation system based on real-time quantification of operation trajectory proposed in this invention.

[0047] Figure 2 This is a flowchart of a puncture teaching simulation method based on real-time quantification of operation trajectory proposed in this invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] In actual clinical puncture surgery, in order to ensure the accuracy and safety of the operation, the following key technical aspects are involved: (1) Calculate the depth and angle of the puncture path based on CT and other sequence medical images; (2) Use auxiliary devices such as robotic arms for high-precision positioning and needle insertion; (3) Use body positioning fixation devices (such as radiotherapy body positioning vacuum bags) to reduce patient movement; (4) Quantify the respiratory cycle through respiratory gating technology to guide the operation in a stable phase; The simulation system and method of the present invention are designed to reproduce, decompose and quantify the training of these key skills in a virtual environment. The specific implementation of the present invention will be described in detail below in conjunction with the key technical aspects.

[0050] Reference Figure 1 A puncture teaching simulation system based on real-time quantification of the operation trajectory, the system comprising:

[0051] The virtual teaching scenario construction module is used to reconstruct a three-dimensional virtual puncture environment based on patient sequence medical images;

[0052] The multimodal interaction and operation acquisition module integrates a force feedback puncture instrument and a spatial positioning device to collect multidimensional spatiotemporal operation data streams generated by trainees during puncture simulation in a three-dimensional virtual puncture environment in real time.

[0053] The operation skill quantification analysis module is connected to the multimodal interaction and operation acquisition module. It is used to preprocess multi-dimensional spatiotemporal operation data streams, extract and fuse quantitative features of spatial accuracy, tissue mechanics identification, physiological motor coordination and operation fluency dimensions, and generate skill profiles.

[0054] The intelligent assessment and adaptive feedback module is used to compare and analyze the trainee's skill profile with pre-stored multi-category standard operating modes to generate assessment results that include deviation information in various dimensions.

[0055] The teaching path dynamic planning module is used to dynamically adjust the subsequent 3D virtual puncture environment based on the student's skill profile and assessment results, thereby generating a personalized training scenario sequence that is adapted to the student's current skill level.

[0056] It should be further explained that, in the specific implementation process, reconstructing the three-dimensional virtual puncture environment includes:

[0057] The image analysis and spatial mapping unit is used to import sequential medical images containing puncture entry point A and target point B. It extracts the boundaries of different tissues through a segmentation algorithm based on a multi-scale feature pyramid network, and maps the two-dimensional pixel coordinates to a unified world coordinate system according to the physical resolution of the images and the inter-layer positional relationship, thus constructing a voxelized three-dimensional tissue model.

[0058] The physiological dynamic simulation unit is used to bind preset dynamic equations to different anatomical structures in a three-dimensional tissue model to simulate tissue deformation and displacement caused by physiological movements such as respiration, and output real-time physiological state signals, which include at least the respiratory phase.

[0059] The risk field mapping unit is used to generate a spatially continuous risk attenuation field in the space of a three-dimensional tissue model, using the surface of the extracted hazardous structures (such as blood vessels and nerves) as the source.

[0060] Specifically, the system loads the patient's sequential medical images (such as DICOM format files from CT or MRI), reads the pixel spacing, slice thickness, and spatial orientation labels of each frame, and establishes an affine transformation matrix from pixel indices (e.g., row indices, column indices, slice indices) to physical world coordinates. A segmentation algorithm based on a multi-scale feature pyramid network is used to automatically identify different tissues. The encoder part of this network consists of multiple stacked convolutional and pooling layers, used to extract and downsample low-resolution feature maps containing global semantic information from the input sequential medical images. The decoder part gradually restores the spatial resolution through upsampling operations, and at each upsampling stage, skip connections are used to connect the feature maps of the same scale corresponding to the intermediate layers of the encoder. Channels are stitched together and then fused through convolutional layers to simultaneously utilize deep semantic information and shallow detail information. The network ultimately outputs a probability map of each pixel belonging to the category of blood vessel, nerve, target lesion, bone, soft tissue, and background. By thresholding and connected component analysis of the category probability map, the three-dimensional mask boundaries of each tissue are obtained. Finally, combined with the known physical resolution (e.g., 0.5 mm per pixel) and interlayer spacing (e.g., 2 mm), the pixel coordinates in the two-dimensional mask are mapped to a right-handed world coordinate system with a certain anatomical landmark of the patient as the origin through affine transformation, generating a parameterized three-dimensional tissue model represented by voxels (three-dimensional pixels), where each voxel records its spatial coordinates, tissue type, and grayscale features inherited from the original image.

[0061] In the constructed 3D tissue model, anatomical structures involved in respiratory motion (such as lung lobes and diaphragm surfaces) are marked as deformable regions. A simplified mass-spring-damped mesh model is bound to each deformable region, where mesh nodes correspond to the centers of voxels within that region, springs connect adjacent nodes, and the damping coefficient simulates tissue viscoelasticity. The core of the dynamic equation is: the mass matrix of a mesh node multiplied by its velocity vector, plus the stiffness matrix multiplied by its displacement vector, equals the external driving force vector. This external driving force vector represents the driving force generated by respiratory motion. To simulate respiration, the external driving force vector is generated by a configurable waveform generator, which outputs a time-varying scalar signal. The signal is a superposition of a sine wave (simulating regular breathing), a ramp function (simulating deep breathing), and random noise (simulating irregular breathing). The generated breathing drive signal is converted into a force vector applied to the mesh nodes of the deformable regions corresponding to key areas such as the diaphragm, thus forming a breathing drive force vector. The dynamic equation is solved by numerical integration (such as the Euler method or the Runge-Kutta method) to calculate the displacement of all mesh nodes at each time step, thereby driving the deformation and displacement of the entire three-dimensional model. At the same time, the breathing phase signal is calculated in real time. The breathing phase signal is a normalized integral value of the breathing drive waveform, which cycles between [0, 2π), where 0 is defined as the end of expiration and π is defined as the end of inspiration. The breathing phase signal is used to provide continuous phase information.

[0062] Extract surface triangular meshes of all hazardous structures (such as blood vessels and nerves) in the space of the 3D tissue model. For any query point P in three-dimensional space, calculate its distance to the triangular mesh on each hazardous structural surface. Shortest Euclidean distance The calculation of the shortest Euclidean distance is achieved by storing all surface triangular patches using a spatial acceleration structure (such as a KD-Tree), and then calculating the distance from the point to the plane containing the nearest neighbor patch after finding the nearest neighbor patch; each hazardous structure type is assigned a preset risk weight. (For example, the risk weight for large arteries is set to 10.0, the risk weight for major nerves is set to 8.0, and the risk weight for small blood vessels is set to 3.0); according to the formula Calculate the risk value of any query point P. In the formula, The summation variable is used to iterate through the triangular meshes of all hazardous structural surfaces; The constant is set to a very small constant (e.g., 1e-6) to avoid division by zero errors; the formula simulates the physical field in which risk decays with the square of the distance; finally, by performing discrete sampling calculations on the entire three-dimensional space, or by using spatial interpolation techniques, a spatially continuous risk scalar field covering the entire virtual surgical area is generated, namely the risk decay field.

[0063] It should be further explained that, during the specific implementation process, the multi-dimensional spatiotemporal operational data stream generated during the puncture simulation in the 3D virtual puncture environment is collected in real time, including:

[0064] The spatial coordinates and posture data of the end of the force feedback puncture instrument are collected by an optical positioning device to form a time-continuous puncture trajectory sequence.

[0065] The six-dimensional force / torque data generated by the interaction between the force feedback puncture instrument simulation and a physical simulation entity model are recorded synchronously to form a mechanical interaction sequence. The physical simulation entity model is a model generated based on a three-dimensional tissue model by assigning physical and mechanical parameters corresponding to the tissue type to its voxels, and is used for physical simulation calculation.

[0066] Detect and record key state transition events during the virtual puncture process, including fascia penetration events, target area entry events, and high-risk invasion events in the risk field, forming a state transition event sequence;

[0067] The puncture trajectory sequence, the mechanical interaction sequence, and the state transition event sequence are synchronized and time-aligned to form a multidimensional spatiotemporal operational data stream. Specifically, time alignment is achieved by using cubic spline interpolation to resample the data points of the puncture trajectory sequence, the mechanical interaction sequence, and the state transition event sequence onto the same equally spaced time axis (e.g., 1ms interval).

[0068] Specifically, one or more infrared optical positioning cameras (such as the OptiTrack series) are deployed above the operating area. A rigid body consisting of at least three non-collinear infrared reflective markers is securely installed at the end of the force feedback puncture instrument. The infrared optical positioning cameras capture the two-dimensional image coordinates of the markers at a frequency of not less than 100Hz. The three-dimensional spatial coordinates of each marker are calculated in real time using the multi-view triangulation principle. Based on the known geometric relationships of the markers, the optimal rigid body transformation (rotation matrix and translation vector) is solved using the singular value decomposition algorithm, thereby obtaining the three-dimensional coordinates of the center point of the tool at the end of the force feedback puncture instrument in the world coordinate system and its orientation represented by quaternions. The orientation data is stored in the order of timestamps to form a puncture trajectory sequence, where each data point is a tuple containing timestamp, position, and orientation.

[0069] Force feedback puncture instruments (such as Geomagic Touch or Simionix products) have built-in high-precision motors and torque sensors. When a student operates the force feedback puncture instrument, the software interface of the instrument reads the torque sensor readings of each joint in real time. Based on the kinematic forward model and dynamic model of the force feedback puncture instrument, it calculates the equivalent six-dimensional force / torque vector acting on the end of the instrument. This vector includes three force components and three torque components, reflecting the simulated resistance and torque applied to the student by the physical simulation entity model through the force feedback algorithm. The six-dimensional force / torque data is collected at a frequency of no less than 100Hz and timestamped to form a mechanical interaction sequence.

[0070] Real-time collision detection and state logic judgment are performed. The triggering condition for a fascia penetration event is: a collision is detected between the tip of the force feedback puncture instrument and a geometry marked as fascia in the physical simulation entity model, and the normal velocity component of the collision exceeds the penetration velocity threshold. The penetration velocity threshold is used in the physical simulation to determine whether the puncture needle has successfully penetrated a layer of tissue (such as fascia), and is set based on the material mechanical properties of the simulated tissue. The triggering condition for a target area entry event is: the tip of the force feedback puncture instrument enters a spherical region with radius r centered on target point B for the first time. The detection process for high-risk invasion events in the risk field is as follows: the risk value of the tip of the force feedback puncture instrument in the risk decay field is calculated in real time. When the risk value is continuously higher than the preset risk alarm threshold and reaches the preset duration, a high-risk invasion event is determined to be triggered, and its duration and peak risk value are recorded. The risk alarm threshold defines the critical risk level for triggering a high-risk invasion event in the risk decay field. Whenever the condition of any event is met, an event record is generated, including the event type and trigger timestamp. The event records are arranged in chronological order to form a state transition event sequence.

[0071] It should be further explained that, in the specific implementation process, quantitative features from spatial accuracy, tissue biomechanics identification, physiological-motor coordination, and operational fluency are extracted and integrated to generate a skill profile, including:

[0072] Based on the puncture trajectory sequence, the dynamic time-normalized distance between it and the planned path is calculated as the path deviation. The cumulative time percentage of the trajectory point falling into the high-value area in the risk decay field (i.e., when the risk value is continuously higher than the preset risk alarm threshold and reaches the preset duration) is counted as the risk assessment index, which together constitute the spatial accuracy feature. Among them, the planned path is generated based on the three-dimensional virtual puncture environment, that is, the path from the entry point A to the target point B.

[0073] Based on the mechanical interaction sequence and state transition event sequence, force curve segments associated with fascia penetration events and target area entry events are identified. The mean of the force rise slope and the fluctuation variance of the force curve segments are calculated as tissue mechanical identification features.

[0074] Based on the instantaneous velocity sequence extracted from the puncture trajectory sequence and the real-time respiratory phase signal output by the physiological dynamic simulation unit, the percentage of displacement occurring within a preset stable phase window in the respiratory cycle is calculated as the physiological motion coordination feature.

[0075] Based on the timeline of the entire operation process, the proportion of effective needle insertion time to total operation time is calculated, and the sample entropy of the needle insertion speed sequence is calculated as a feature of operation smoothness.

[0076] The calculated spatial precision features, tissue biomechanics identification features, physiological motor coordination features, and operational fluency features are normalized and weighted and fused to output a multi-dimensional skill feature vector, which represents the skill profile of a single operation.

[0077] Specifically, the planned path (theoretical puncture path) from entry point A to target point B is obtained, represented by a series of ordered three-dimensional path points. For the collected actual puncture trajectory sequence, its positional components are extracted to obtain the actual path points. The path deviation between the actual path points and the three-dimensional path points is calculated using a dynamic time warping algorithm. The specific steps include: 1) Calculating the Euclidean distance matrix between all pairs of actual paths and three-dimensional path points; 2) Calculating the cumulative distance matrix using a dynamic programming recursive formula based on the Euclidean distance matrix; 3) Finding the optimal curved path by backtracking from the endpoint of the cumulative distance matrix; 4) The total cumulative distance of the optimal curved path is the path deviation.

[0078] The calculation of the risk assessment index includes: for each actual path point in the trajectory sequence, querying the risk value in the risk decay field of its location; counting all trajectory points whose risk value is greater than a preset judgment threshold (used to define high-value areas when quantifying the risk assessment index, such as setting the judgment threshold to 60% of the maximum risk value); calculating the percentage of the total time (number of points multiplied by the sampling interval) covered by high-risk trajectory points to the total operation time, and this percentage is used as the risk assessment index;

[0079] Based on the synchronized state transition event sequence, the time points of the fascia penetration event and the target area entry event are located. In the mechanical interaction sequence, the three-dimensional resultant force magnitude curves are extracted within time windows of extended feature analysis window lengths (e.g., 200ms) before and after the event occurrence time point to obtain force curve segments. Savitzky-Golay filters are applied to the force curve segments to smooth them and remove high-frequency noise. The smoothed force curves are numerically differentiated to calculate their first derivative, i.e., the force rate of change curve. A short analysis window (e.g., 50ms) is selected before and after the event occurrence time point, and the arithmetic mean of the force rate of change sequence within the window is calculated as the mean of the force rise slope. At the same time, the standard deviation of the force rate of change sequence within the window is calculated as the force fluctuation variance, which is used to measure the stability of the penetration action.

[0080] Real-time respiratory phase signals are acquired from the physiological dynamic simulation unit, which periodically change between [0, 2π). A stable phase window is set, for example, respiratory phase values ​​∈ [5π / 4, 7π / 4] (corresponding to the relatively stable period from the end of expiration to the beginning of inspiration). For the puncture trajectory sequence, the displacement vector within each sampling interval is calculated, and the instantaneous velocity is obtained by taking the modulus. During the effective needle insertion period of the entire operation, i.e., from the time when the force feedback puncture instrument first contacts the virtual skin to the time when it enters the target area, each instantaneous velocity value collected during this period is accumulated (integrated) to obtain the total displacement generated during the entire needle insertion process. At the same time, only the moment when the respiratory phase is within the preset stable window is selected, and the instantaneous velocity corresponding to this moment is accumulated (integrated) separately to obtain the displacement completed within the respiratory stable window. The ratio of the displacement within the respiratory stable window to the total displacement is the physiological motion coordination characteristic.

[0081] Calculating the proportion of effective needle insertion time to total operation time includes: accumulating the time during which the instantaneous speed is greater than the minimum effective speed threshold, and dividing by the total operation time; where the minimum effective speed threshold is a preset lower limit value used to distinguish between effective and purposeful puncture needle insertion movements and ineffective pauses or preparatory movements. Time periods when the instantaneous speed is lower than the minimum effective speed threshold are determined not to be effective needle insertion processes. This threshold is set based on empirical data from actual puncture operations.

[0082] The calculation of the sample entropy of the needle insertion velocity sequence includes: taking the instantaneous velocity sequence, setting the reference vector length and similarity tolerance threshold for comparison; where the reference vector length refers to the number of consecutive instantaneous velocity values ​​used to be truncated and compared when calculating the sample entropy; the similarity tolerance threshold refers to the maximum range of differences between corresponding velocity values ​​allowed when judging whether two velocity vector templates are similar, and this threshold is set as a fixed proportion of the standard deviation of the entire instantaneous velocity sequence; sequentially extracting all subsequences in the instantaneous velocity sequence that conform to the reference vector length as templates, and counting the number of matches between all templates where the difference between their corresponding data points is less than the similarity tolerance threshold; similarly, counting the number of matches between all subsequence templates whose vector length is increased by one unit; based on the relationship between the number of matches for the two lengths and the total number of templates, the sample entropy value is calculated; where the sample entropy value is used to quantify the complexity and irregularity of the velocity sequence, the lower the value, the smoother and more fluid the motion process.

[0083] It should be further explained that, in the specific implementation process, the trainee's skill profile is compared and analyzed with pre-stored multi-category standard operating procedures to generate an evaluation result containing deviation information in various dimensions, including:

[0084] Receive skill profiles; input the skill feature vectors generated by the trainees' operations into a knowledge base that has multiple sets of standard operation mode feature vectors. The construction method is as follows: after extracting skill feature vectors from a large amount of expert operation data, use a clustering algorithm to divide them into several sets. Each set is represented by a central feature vector and the distribution covariance matrix of the feature vectors in the set. Each set corresponds to a standard operation mode.

[0085] Calculate the Euclidean distance between the skill feature vector of the trainee's operation and the center of the set of feature vectors of each standard operation mode in the knowledge base in the feature space composed of the dimensions of spatial precision, organizational mechanical identification, physiological motor coordination and operation fluency.

[0086] The trainee's operation mode category is determined based on the minimum distance principle. The evaluation deviation value of each dimension of the trainee's skill feature vector is obtained by subtracting the value of each dimension of the central feature vector of the corresponding standard operation mode from the value of each dimension of the skill feature vector.

[0087] Based on the assessment deviation values ​​of each dimension, a dynamic feedback strategy is generated, which triggers targeted visual, tactile, or auditory feedback instructions: when the assessment deviation value of the spatial accuracy dimension exceeds the preset spatial accuracy deviation threshold, an instruction to enhance the visualization of the risk field is triggered; when the assessment deviation value of the tissue biomechanics identification dimension exceeds the preset tissue biomechanics identification deviation threshold, an instruction to provide comparative tactile guidance in subsequent operations is triggered; when the assessment deviation value of the physiological motor coordination dimension exceeds the preset physiological motor coordination deviation threshold, an instruction to highlight the breathing phase window in the interface is triggered. Among them, the spatial accuracy deviation threshold is a key indicator used to measure the degree of deviation of the trainee in the spatial accuracy dimension, the tissue biomechanics identification deviation threshold is used to determine whether the trainee deviates from the normal range in the tissue biomechanics identification dimension, and the physiological motor coordination deviation threshold is used to determine whether the trainee's deviation in the physiological motor coordination dimension is significant.

[0088] It should be further explained that, in the specific implementation process, the subsequent 3D virtual puncture environment is dynamically adjusted to generate a personalized training scenario sequence adapted to the trainee's current skill level, including:

[0089] Based on the trainee's historical skill profile and their assessment deviations in spatial accuracy, biomechanical identification, physiological-motor coordination, and operational fluency, the skill development trend is analyzed. Specifically, this analysis involves: using linear regression to fit a trend line of the most recent N (e.g., 20) training scores for that dimension (which can be represented by the corresponding dimension value or a comprehensive score in the skill profile), and calculating the slope; a positive slope indicates progress, a negative slope indicates regression, and a slope close to zero indicates a plateau; and calculating the standard deviation or coefficient of variation of the most recent M (e.g., 5) training scores for that dimension to assess performance fluctuations; and combining this with the assessment deviations, if a dimension exhibits a long-term flat trend (the absolute value of the slope is less than a preset slope threshold), and the recent assessment deviations consistently exceed the corresponding thresholds (spatial accuracy deviation threshold, biomechanical identification deviation threshold, physiological-motor coordination deviation threshold), then that dimension is determined to be the current skill bottleneck.

[0090] Based on the analysis of skill development trends, a personalized training scenario sequence is generated, consisting of multiple 3D virtual puncture environments with different scenario parameter configurations. Each training scenario in this sequence matches or challenges the trainee's current skill level by adjusting at least one scenario parameter. The scenario parameters include anatomical complexity, physiological motion interference intensity, and risk field distribution. Specifically, anatomical complexity involves adjusting the distribution density and tortuosity of dangerous structures in the 3D tissue model within the target area and around the path; physiological motion interference intensity involves adjusting the amplitude, frequency, and irregular components of the respiratory waveform in the physiological dynamic simulation unit; and risk field distribution involves adjusting the value of the risk weight in the risk decay field or the exponential parameter in the risk decay formula to change the gradient and range of the risk field.

[0091] If a bottleneck is identified in a certain dimension, then in the next scenario, the challenge level of the scenario parameters most relevant to that dimension will be increased in a targeted manner. For example, for the bottleneck of spatial accuracy, the number of blood vessels will be increased or reduced to make the path more tortuous, and the value of the formula will be reduced (to expand the risk field range and make the gradient gentler), increasing the risk of misjudgment; for the bottleneck of mechanical identification, in the tissue mechanical model, the difference in elastic modulus between different tissue layers (such as fat and muscle) will be reduced to make the tactile difference more obvious, increasing the identification difficulty; for the bottleneck of physiological coordination, the respiratory rate will be increased to shorten the stability window, or the respiratory amplitude will be deepened to expand the target displacement, or the noise will be increased to introduce irregular interference.

[0092] If a trainee's assessment deviation value in any dimension exceeds the preset deviation threshold for that dimension, and the skill development trend analysis based on historical skill profiles indicates that the dimension represents a skill bottleneck, then in subsequent personalized training scenario sequences, the challenge level of the scenario parameters (anatomical complexity, physiological and motor interference intensity, risk field distribution, or one or more of these) corresponding to that dimension will be gradually increased. If a trainee's assessment deviation value in all dimensions is lower than the preset deviation threshold for that dimension, and the skill development trend analysis based on historical skill profiles indicates that the performance in each dimension is stable or shows a progressive trend, then a comprehensive high-level challenge scenario integrating high anatomical complexity, strong physiological interference, and complex risk field configuration will be generated and arranged as a subsequent component of the personalized training scenario sequence.

[0093] Reference Figure 2 A puncture teaching simulation method based on real-time quantification of the operation trajectory includes:

[0094] S1. Reconstruct a three-dimensional virtual puncture environment based on the patient's sequence medical images;

[0095] S2. Real-time acquisition of multi-dimensional spatiotemporal operation data streams generated when trainees perform puncture simulations in a three-dimensional virtual puncture environment;

[0096] S3. Preprocess the multidimensional spatiotemporal operation data stream, extract and fuse quantitative features of spatial accuracy, tissue mechanics identification, physiological motion coordination and operation fluency dimensions to generate a skill profile;

[0097] S4. Compare and analyze the trainee's skill profile with the pre-stored multi-category standard operating modes to generate an evaluation result that includes deviation information in each dimension.

[0098] S5. Based on the trainee's skill profile and assessment results, dynamically adjust the subsequent three-dimensional virtual puncture environment to generate a personalized training scenario sequence that suits the trainee's current skill level.

[0099] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. The focus of each embodiment is on its differences from other embodiments. In particular, the apparatus embodiments are described simply because they are fundamentally based on the method embodiments; relevant details can be found in the descriptions of the method embodiments.

[0100] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0105] Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0106] In conclusion, the above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A puncture teaching simulation system based on real-time quantification of an operation trajectory, characterized by, include: The virtual teaching scenario construction module is used to reconstruct a three-dimensional virtual puncture environment based on patient sequence medical images; A multimodal interaction and operation acquisition module is used to acquire in real time the multidimensional spatiotemporal operation data stream generated by the trainee during puncture simulation in a three-dimensional virtual puncture environment; wherein, the real-time acquisition of the multidimensional spatiotemporal operation data stream generated by the trainee during puncture simulation in a three-dimensional virtual puncture environment includes: The spatial coordinates and attitude data of the end of the force feedback puncture instrument are collected by an optical positioning device to form a time-continuous puncture trajectory sequence; six-dimensional force / torque data generated by the interaction between the force feedback puncture instrument simulation and a physical simulation entity model are recorded simultaneously to form a mechanical interaction sequence; key state transition events during the virtual puncture process are detected and recorded, including fascia penetration events, target area entry events, and high-risk invasion events in the risk field, to form a state transition event sequence; the puncture trajectory sequence, mechanical interaction sequence, and state transition event sequence are synchronized in time to form a multi-dimensional spatiotemporal operational data stream. The operational skill quantification analysis module is used to preprocess multi-dimensional spatiotemporal operational data streams, extracting and fusing quantitative features from spatial accuracy, tissue biomechanics identification, physiological-motor coordination, and operational fluency dimensions to generate a skill profile. Spatial accuracy features include the dynamic time-normalized distance calculated based on the puncture trajectory sequence and the planned path as path deviation, and the cumulative time percentage of trajectory points falling into high-value areas of the risk decay field as a risk assessment index. Tissue biomechanics identification features include the mean and variance of the force rise slope calculated based on force curve segments associated with fascial penetration events and target area entry events. Physiological-motor coordination features include the percentage of displacement occurring within a preset stable phase window of the respiratory cycle, calculated based on the instantaneous puncture velocity sequence and real-time respiratory phase signals. Operational fluency features include the proportion of effective needle insertion time to total operation time and the sample entropy of the needle insertion velocity sequence. The extraction and fusion of quantitative features to generate a skill profile includes: based on the puncture trajectory sequence, calculating the dynamic time regularization distance between it and the planned path as the path deviation, and calculating the cumulative time percentage of trajectory points falling into the high-value area in the risk decay field as the risk assessment index, which together constitute the spatial accuracy feature. Based on the mechanical interaction sequence and state transition event sequence, force curve segments associated with fascia penetration events and target area entry events are identified. The mean of the force rise slope and the fluctuation variance of the force curve segments are calculated as tissue mechanical identification features. Based on the instantaneous velocity sequence extracted from the puncture trajectory sequence and the real-time respiratory phase signal output by the physiological dynamic simulation unit, the percentage of displacement occurring within a preset stable phase window in the respiratory cycle is calculated as the physiological motion coordination feature. Simultaneously, the proportion of effective needle insertion time to total operation time is calculated, and the sample entropy of the needle insertion speed sequence is calculated as a feature of operation smoothness. All calculated features are normalized and weighted and fused to output a multi-dimensional skill feature vector, which represents the skill profile of a single operation. The intelligent assessment and adaptive feedback module is used to compare and analyze the trainee's skill profile with pre-stored multi-category standard operating modes to generate assessment results that include deviation information in various dimensions. The teaching path dynamic planning module is used to dynamically adjust the subsequent 3D virtual puncture environment based on the student's skill profile and assessment results, thereby generating a personalized training scenario sequence that is suitable for the student's current skill level. It also combines the skill bottlenecks identified by trend analysis of the student's historical skill profile across various dimensions to dynamically adjust the challenge level of at least one scenario parameter in the subsequent 3D virtual puncture environment. These scenario parameters include anatomical complexity, physiological motion interference intensity, and risk field distribution, thereby generating a personalized training scenario sequence that is suitable for the student's current skill level and provides targeted reinforcement. 2.The puncture teaching simulation system based on operation trajectory real-time quantification according to claim 1, characterized in that: Reconstructing a three-dimensional virtual puncture environment includes: The image analysis and spatial mapping unit is used to import sequential medical images containing puncture entry point A and target point B. It extracts the boundaries of different tissues through a segmentation algorithm based on a multi-scale feature pyramid network, and maps the two-dimensional pixel coordinates to a unified world coordinate system according to the physical resolution of the images and the inter-layer positional relationship, thus constructing a voxelized three-dimensional tissue model. The physiological dynamic simulation unit is used to bind preset dynamic equations to different anatomical structures in a three-dimensional tissue model to simulate tissue deformation and displacement caused by physiological movement, and output real-time physiological state signals, which include at least the respiratory phase. The risk field mapping unit is used to generate a spatially continuous risk attenuation field in the space of a three-dimensional organizational model, using the surface of the extracted hazardous structure as the source.

3. The puncture teaching simulation system based on real-time quantification of operation trajectory according to claim 2, characterized in that, The segmentation algorithm of the multi-scale feature pyramid network is as follows: a network structure is established, which includes an encoder and a decoder. The encoder is used to quickly downsample and extract global context information, and the decoder performs multi-level feature fusion. The decoder concatenates and convolves the feature maps of different scales of the encoder with the upsampled feature maps.

4. The puncture teaching simulation system based on real-time quantification of operation trajectory according to claim 1, characterized in that, The specific method for detecting and recording high-risk intrusion events in the risk field is as follows: calculate the risk value of the puncture instrument tip in the risk decay field in real time. When the risk value is continuously higher than the preset risk alarm threshold and reaches the preset time, a high-risk intrusion event is determined to be triggered, and the corresponding area is recorded as the high-value area.

5. The puncture teaching simulation system based on real-time quantization of operation trajectory according to claim 1, characterized in that, The method for calculating the mean of the force rise slope and the variance of the fluctuation includes: after smoothing and filtering the force curve segment, obtaining its first time derivative, taking the arithmetic mean of the derivative sequence within a preset time window before and after the event occurrence time as the mean of the force rise slope, and calculating the standard deviation of the derivative sequence within the window as the variance of the force fluctuation.

6. The puncture teaching simulation system based on real-time quantization of operation trajectory according to claim 1, characterized in that, The trainee's skill profile is compared and analyzed with pre-stored multi-category standard operating procedures to generate an assessment result that includes deviation information for each dimension, including: Receive skill profiles; input the skill feature vectors generated by the student's operations into a knowledge base that has multiple sets of feature vectors for standard operation modes stored in advance; Calculate the Euclidean distance between the skill feature vector of the trainee's operation and the center of the set of feature vectors of each standard operation mode in the knowledge base in the feature space composed of the dimensions of spatial precision, organizational mechanical identification, physiological motor coordination and operation fluency. The trainee's operation mode category is determined based on the minimum distance principle. The evaluation deviation value of each dimension of the trainee's skill feature vector is obtained by subtracting the value of each dimension of the central feature vector of the corresponding standard operation mode from the value of each dimension of the skill feature vector. Based on the evaluation deviation values ​​of each dimension, a dynamic feedback strategy is generated, which triggers targeted visual, force, or auditory feedback instructions.

7. The puncture teaching simulation system based on real-time quantization of operation trajectory according to claim 1, characterized in that, The system dynamically adjusts the subsequent 3D virtual puncture environment to generate a personalized training scenario sequence adapted to the trainee's current skill level, including: Based on the trainees' historical skill profiles and their evaluation deviations in various dimensions such as spatial accuracy, organizational biomechanics identification, physiological-motor coordination, and operational fluency, we analyze the trends in skill development. Based on the analysis of skill development trends, a personalized training scenario sequence is generated, consisting of multiple three-dimensional virtual puncture environments with different scenario parameter configurations. Each training scenario in the sequence is adjusted by adjusting at least one scenario parameter to match or challenge the trainee's current skill level. If a trainee's assessment deviation value in any dimension exceeds the preset threshold for that dimension, and the skill development trend analysis shows that the dimension represents a skill bottleneck, then in subsequent personalized training scenario sequences, the challenge level of the scenario parameters corresponding to that dimension will be increased. If a trainee's assessment deviation value in all dimensions is lower than the preset threshold for the corresponding dimension, and the skill development trend analysis shows that the performance in each dimension is stable or improving, then a comprehensive high-level challenge scenario will be generated and arranged as a subsequent component of the personalized training scenario sequence.

8. A puncture teaching simulation method based on real-time quantification of the operation trajectory, characterized in that, The method, applied to a puncture teaching simulation system based on real-time quantization of operation trajectory as described in any one of claims 1-7, comprises: S1. Reconstruct a three-dimensional virtual puncture environment based on patient sequence medical images; S2. Real-time acquisition of multi-dimensional spatiotemporal operation data streams generated when trainees perform puncture simulations in a three-dimensional virtual puncture environment; S3. Preprocess the multidimensional spatiotemporal operation data stream, extract and fuse quantitative features of spatial accuracy, tissue mechanics identification, physiological motion coordination and operation fluency dimensions to generate a skill profile; S4. Compare and analyze the trainee's skill profile with the pre-stored multi-category standard operating modes to generate an evaluation result that includes deviation information in each dimension. S5. Based on the trainee's skill profile and assessment results, dynamically adjust the subsequent three-dimensional virtual puncture environment to generate a personalized training scenario sequence that suits the trainee's current skill level.

Images