Clinical surgery experiment virtual simulation teaching method, device, equipment and medium

Abstract

The invention relates to a clinical surgery experiment virtual simulation teaching method and device, equipment and a medium. The method comprises the following steps: responding to an obtained teaching task configuration file; the teaching task configuration file comprises a clinical scene and an evaluation standard; calling a corresponding three-dimensional digital physiological model according to the teaching task configuration file, and performing special effect rendering according to the clinical scene to obtain a visual virtual simulation experiment scene; responding to the obtained user action data, performing real-time deformation on the three-dimensional digital physiological model, and generating feedback parameters; the feedback parameter is used for indicating the interaction control to work according to the feedback parameter; and comparing the user action data with preset standard expert action data, and generating a teaching evaluation result based on an evaluation standard. By adopting the method, high fidelity can be carried out on the whole process of clinical surgery experiment teaching, effective technical support is provided for medical skill teaching, and the method is suitable for multi-scene and multi-level medical teaching practice requirements.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical teaching, and particularly relates to a clinical surgery experiment virtual simulation teaching method, device, equipment and medium. BACKGROUND

[0002] With the development of medical education and clinical surgery training technology, emerging teaching methods such as virtual reality (VR), augmented reality (AR) and digital twin have appeared. Such technologies provide medical students and junior surgeons with safer, controllable and highly repeatable training scenarios by constructing an immersive three-dimensional interactive environment, significantly improving the problem of heavy dependence on real cases and limited training opportunities in traditional teaching.

[0003] In the prior art, clinical surgery experiment teaching mainly relies on animal experiments, cadaver dissection or limited clinical practice opportunities. The simulators introduced by some colleges are mostly physical models, which can usually only simulate a single operation and are difficult to realize system simulation of complete surgical procedures and surgical processes.

[0004] Using a general static simulation model, it is difficult to reflect clinical complex conditions such as lesion structure, tissue variation or bleeding, resulting in limited teaching coverage. Although surface collision detection is supported, it cannot simulate physiological responses such as real bleeding, compression and perfusion, and the simulation effect is poor. SUMMARY

[0005] Therefore, it is necessary to provide a clinical surgery experiment virtual simulation teaching method, device, equipment and medium capable of fine simulation to meet practical teaching in view of the above technical problems.

[0006] In a first aspect, the application provides a clinical surgery experiment virtual simulation teaching method, comprising:

[0007] in response to the obtained teaching task configuration file; the teaching task configuration file includes a clinical scene and an evaluation standard;

[0008] calling a corresponding three-dimensional digital physiological model according to the teaching task configuration file, and performing special effect rendering according to the clinical scene to obtain a visual virtual simulation experiment scene;

[0009] in response to the obtained user action data, deforming the three-dimensional digital physiological model in real time and generating feedback parameters; the feedback parameters are used to instruct the interactive control to work according to the feedback parameters;

[0010] comparing the user action data with preset standard expert action data, and generating a teaching evaluation result based on the evaluation standard.

[0011] In one of the embodiments, the three-dimensional digital physiological model is constructed by the following method:

[0012] Obtaining a medical anatomy image set; the medical anatomy image set includes medical anatomy images corresponding to a plurality of clinical scenarios respectively;

[0013] Performing body data extraction on the medical anatomy images to obtain a body data set;

[0014] Classifying the medical anatomy images into tissue organs using a segmentation algorithm, and combining the body data to generate tissue organ types and corresponding binary masks and confidence levels;

[0015] Performing isosurface extraction on the binary masks to obtain a triangular facet mesh; the triangular facet mesh includes a plurality of mesh nodes and corresponding node coordinates;

[0016] Obtaining corresponding tissue organ attributes according to the tissue organ types, and combining the triangular facet mesh to generate a finite element mesh; the tissue organ attributes include elasticity, tension, and viscosity;

[0017] Performing model deformation and finite element analysis of physiological signals according to the finite element mesh to obtain the three-dimensional digital physiological model.

[0018] In one of the embodiments, performing model deformation and finite element analysis of physiological signals according to the finite element mesh to obtain the three-dimensional digital physiological model, including:

[0019] Obtaining a stiffness matrix and a mass matrix according to the finite element mesh;

[0020] Obtaining a deformation mode according to the stiffness matrix and the mass matrix;

[0021] Based on the Windkessel model, calculating a blood vessel parameter curve according to the finite element mesh; the blood vessel parameters include blood pressure and flow rate;

[0022] Mapping the blood vessel parameter curve to the stiffness matrix to obtain a local rupture bleeding rate function;

[0023] Fusing the deformation mode, the blood vessel parameter curve, and the bleeding rate function to obtain the three-dimensional digital physiological model.

[0024] In one of the embodiments, the corresponding three-dimensional digital physiological model is called according to a teaching task configuration file, and special effect rendering is performed according to the clinical scenario to obtain a visual virtual simulation experiment scene, including:

[0025] Performing material rendering on the three-dimensional digital physiological model to obtain a simulation model;

[0026] According to the local rupture position corresponding to the clinical scenario, performing particle emission rendering on the simulation model using the bleeding rate function to obtain a dynamic model;

[0027] Map the dynamic model to the preset environment to obtain a visual virtual simulation experiment scene.

[0028] In one of the embodiments, in response to the obtained user action data, the three-dimensional digital physiological model is deformed in real time, and feedback parameters are generated, including:

[0029] Based on the Unity engine, the collision interaction event between the user and the three-dimensional digital physiological model is obtained according to the user action data;

[0030] The time-based penetration depth, position data and posture data of the user action data are extracted;

[0031] The penetration depth, position data and posture data are input into a deformation mode to obtain linear superposition deformation data of the corresponding three-dimensional digital physiological model;

[0032] According to the penetration depth and the corresponding time sequence, the mechanical feedback data of the three-dimensional digital physiological model are calculated;

[0033] The feedback parameters are generated according to the mechanical feedback data; the feedback parameters include vibration parameters.

[0034] In one of the embodiments, the user action data is compared with the preset standard expert action data, and a teaching evaluation result is generated based on an evaluation standard, including:

[0035] The corresponding standard expert action data is obtained according to the collision interaction event; the standard expert action data includes an expert template trajectory;

[0036] The student experiment trajectory vector is obtained according to the time-based position data and posture data;

[0037] The student experiment trajectory vector is compared with the expert template trajectory based on a preset speed tolerance error window of dynamic time rule to obtain a trajectory error;

[0038] The teaching evaluation result is obtained according to the trajectory error based on the evaluation standard; the teaching evaluation result includes an accuracy score.

[0039] In one of the embodiments, the trajectory error is obtained by the following formula:

[0040]

[0041] Wherein, RMSE is the trajectory error; T is the total time length of the trajectory; post[t] is the student experiment trajectory vector of the t frame; pos expert [t] is the expert template trajectory of the t frame.

[0042] In a second aspect, the application also provides a clinical surgical experiment virtual simulation teaching device, comprising:

[0043] The teaching experiment task configuration module is configured to respond to the acquired teaching task configuration file; the teaching task configuration file includes a clinical scene and an evaluation standard;

[0044] The simulation module is configured to call a corresponding three-dimensional digital physiological model according to the teaching task configuration file, and perform special effect rendering according to the clinical scene to obtain a visual virtual simulation experiment scene;

[0045] The interaction module is configured to respond to the acquired user action data, deform the three-dimensional digital physiological model in real time, and generate feedback parameters;

[0046] The teaching evaluation module is configured to compare the user action data with preset standard expert action data, and generate a teaching evaluation result based on the evaluation standard.

[0047] In a third aspect, the present application also provides a computer device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of any of the above-mentioned clinical surgical experiment virtual simulation teaching methods when executing the computer program.

[0048] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of any of the above-mentioned clinical surgical experiment virtual simulation teaching methods when executed by a processor.

[0049] The above-mentioned clinical surgical experiment virtual simulation teaching method, device, equipment and medium can realize all-weather, repetitive and high-safety training mode by calling a three-dimensional digital physiological model adapted to a teaching task and combining dynamic special effect rendering of a clinical specific scene, can avoid ethical and resource limited problems, can build a surgical training environment with strong reality, rich details and interaction for students, and can help to improve cognitive immersion and learning efficiency. The real-time action data of the user can be responded to, the modal deformation algorithm and the mechanical feedback calculation model are combined to realize local deformation and feedback control of the three-dimensional model, and the cognitive of the students on tactile force in surgical operations such as tissue cutting, traction and pressing is enhanced. The real-time collection of the user operation track is supported, and the user operation track can also be compared with a standard expert track in multiple dimensions, and a score result can be output according to the evaluation standard, so as to provide an objective, automatic and explainable evaluation basis for teaching feedback and skill examination. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiments or related art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0051] Figure 1 A flowchart of a clinical surgical experiment virtual simulation teaching method of the present application is shown in the figure.

[0052] Figure 2 A flowchart of a sub-step of step S102 is shown in the figure.

[0053] Figure 3 A flowchart of a sub-step of step S104 is shown in the figure.

[0054] Figure 4 A block diagram of a clinical surgical experiment virtual simulation teaching device of the present application is shown in the figure. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0056] In one embodiment, as shown in Figure 1 , a clinical surgical experiment virtual simulation teaching method is provided, and the present embodiment takes the method applied to a terminal as an example. It should be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is realized through the interaction of the terminal and the server. In the present embodiment, the method includes the following steps:

[0057] S101, in response to the obtained teaching task configuration file; the teaching task configuration file includes a clinical scene and an evaluation standard.

[0058] Illustratively, the teaching task configuration file can be generated by a teacher or a teaching platform with a set of structured parameters, including but not limited to target procedure, lesion type, simulation scene number, task duration, interactive device parameters, and skill index system for evaluation. Optionally, the teaching task configuration file can be generated by manual interactive setting in the teaching background or automatically generated by algorithm according to the learning progress of the student. Illustratively, when the teaching task is configured as gallbladder resection surgery training, the file will specify that the scene should load a digital anatomy model containing characteristics of gallstones or cholecystitis, and set evaluation indicators such as incision accuracy, clamp path, resection time, and injury rate. Among them, the clinical scene refers to a digital simulation environment that closely matches the actual surgical task, usually corresponding to one or more clinical surgical operations, including laparoscopic appendectomy, open suture training, partial hepatectomy, or common bile duct exploration, etc., with real spatial structure, pathological characteristics and interactive feedback requirements. The evaluation standard refers to the standard index system for quantitative evaluation of student operation behavior according to the teaching goal, which can include accuracy, timeliness, stability and other basic operation dimensions, and can also set corresponding scoring rules combined with clinical risk points in specific surgical procedures.

[0059] S102, according to the teaching task configuration file, call the corresponding three-dimensional digital physiological model, and perform special effect rendering according to the clinical scene to obtain a visual virtual simulation experiment scene.

[0060] Illustratively, according to the scene identification information in the teaching task configuration file, the corresponding three-dimensional digital physiological model is called, and a complete visual virtual simulation experiment scene is constructed. The three-dimensional digital physiological model is derived from clinical CT (Computed Tomography, electronic computed tomography) or MRI (Magnetic Resonance Imaging, magnetic resonance imaging) image data, and after organ region segmentation by deep learning algorithm, a complete structure three-dimensional entity model is generated by surface reconstruction and grid optimization algorithm. Optionally, in the construction process, corresponding pathological feature annotations can be added to improve the clinical authenticity of the teaching content. The called model not only contains geometric structure and texture characteristics, but also integrates finite element mechanics attributes and dynamic physiological curves, including tissue elasticity, bleeding rate, breathing driven displacement, etc. Further, the model is loaded in the simulation engine, and combined with the specified clinical scene parameters for special effect rendering, a virtual experiment environment with real lighting, tissue material, depth of surgical field and interactive hotspots is formed.

[0061] S103, in response to the acquired user action data, deforming the three-dimensional digital physiological model in real time, and generating feedback parameters; the feedback parameters are used to indicate that the interactive control works according to the feedback parameters.

[0062] Illustratively, the user action data refers to the operation behavior record generated by the trainee through the interactive interface such as VR handle, force feedback gloves, space positioning device, etc. in the training process, including instrument displacement trajectory, rotation posture, operation sequence, contact depth, speed change, etc. Specifically, the user action data is transmitted into the physical simulation module in real time through high-frequency sampling, and the deformation of the corresponding tissue area is calculated according to the finite element model constructed in advance, so as to realize the dynamic display of the operation effect such as cutting, pressing, tearing, suturing, etc. Since the digital tissue has material properties and dynamic parameters, feedback parameters can be generated based on the node stress state and deformation response, which can include the size of the reaction force, the tissue rebound speed, the damping coefficient of the haptic, etc. The feedback parameters will drive the interactive control to work in real time, for example, control the vibration frequency and amplitude of the handle, the stress distribution feedback of the gloves, or the in-situ sound and visual prompt, so that the trainee can obtain a more realistic interactive experience.

[0063] S104, compare the user action data with the preset standard expert action data, and generate a teaching evaluation result based on the evaluation standard.

[0064] Illustratively, the user action data collected is compared with the preset standard expert action data, and a teaching evaluation result is generated based on the evaluation standard. The expert action data can refer to the operation task completed by the teacher in the same scene, and its trajectory, force curve, timing parameter, etc. are stored in the platform as a teaching paradigm. Further, in the comparison, trajectory alignment, root mean square error calculation, angle deviation analysis, force correlation evaluation, etc. can be used to comprehensively determine the deviation degree, operation consistency and key skill completion quality of the trainee's operation. Optionally, the teaching evaluation result can be output in the form of score, grade, prompt or graphical report for the teacher to review or feedback to the trainee.

[0065] In the above-mentioned virtual simulation teaching method for clinical surgery experiments, by introducing a teaching task configuration file as a control entry, a matched three-dimensional digital physiological model and an evaluation template can be automatically called according to different clinical scenarios, thereby realizing high decoupling and dynamic matching of teaching content and simulation resources, improving the teaching task adaptability and expansion capability of the simulation platform, and avoiding the problems of traditional simulation systems, such as dependence on manual configuration, fixed models, and poor expansion. Based on the preset clinical scenario, special effect rendering processing is performed, so that the three-dimensional digital model has dynamic blood flow, tissue lesion, and surgical lighting expression capabilities, thereby constructing a visual environment closer to the real operating room, significantly improving the immersion and task substitution of the students, and helping to form a perception of the real anatomical environment. The physical deformation of the three-dimensional model is driven by user action data, and feedback parameters are generated in real time to simulate the tactile changes and physiological feedback of the tissue, significantly improving the students' ability to identify and force control the tactile differences of different tissue structures, and effectively making up for the lack of operation feeling in traditional video teaching. The user operation trajectory and the expert standard action trajectory are compared in multiple dimensions, and the evaluation standard is automatically output to output the teaching score results, not only can the students obtain precise feedback, but also the skill level in the teaching process has objective and traceable quantitative basis, breaking through the limitations of traditional subjective scoring method, and improving the fairness and scientificity of teaching evaluation.

[0066] In one of the embodiments, the three-dimensional digital physiological model is constructed by the following method:

[0067] S11, acquire a set of medical anatomical images; the set of medical anatomical images includes medical anatomical images corresponding to a plurality of clinical scenarios respectively.

[0068] Illustratively, the set of medical anatomical images can be obtained from a public database that has been processed for privacy protection, a clinical cooperation hospital, or an internal collection platform. The medical anatomical images can include CT, MRI, or ultrasound image sequences, and are required to have a certain resolution, annotation quality, and interlayer consistency, and can cover key tissue organs of the human body. The set of medical anatomical images can correspond to different clinical teaching scenarios according to their clinical characteristics and lesion types, such as gallbladder stone corresponding to gallbladder resection training, and gastric perforation corresponding to repair and suture training. Further, when collecting or selecting image data, cases with pathological characteristics are given priority to enhance the teaching coverage and clinical fit of the model.

[0069] S12, extract body data from the medical anatomical images to obtain a set of body data.

[0070] Illustratively, the medical anatomy image is stored in the DICOM (Digital Imaging and Communications in Medicine) format, and the MONAI (Medical Open Network for AI) is used to extract the volume data from the continuous slices of the medical anatomy image to obtain a volume data set containing voxel distribution information. Specifically, the volume data extraction is to reconstruct the multi-layer image slices into a three-dimensional data structure according to the original spatial direction, so as to form a three-dimensional volume data matrix with voxel density labels, and to retain the volume, surface area and spatial relationship between the anatomical structures.

[0071] S13, using a segmentation algorithm to classify the medical anatomy image into tissue organs, and combining the volume data to generate tissue organ types and corresponding binary masks and confidence.

[0072] Illustratively, the U-Net (U-Net Convolutional Neural Network) is used to perform layer-by-layer discrimination on the input volume data, so as to output a probability distribution map of each type of organ, convert the probability map into an explicit tissue organ type label map according to the maximum probability method, and extract the corresponding binary mask. Each mask region represents the position distribution of a certain organ in the image. Optionally, when constructing a three-dimensional digital physiological model containing lesions, the lesion morphology of the tissue organ in the binary mask is the target region, and the corresponding binary mask is the reserved value 1.

[0073] S14, extracting the isosurface of the binary mask to obtain a triangular facet mesh; the triangular facet mesh includes a plurality of mesh nodes and corresponding node coordinates.

[0074] Illustratively, the Marching Cubes (Marching Cubes) or Dual Contouring (Dual Contouring) isosurface algorithm or the like is used to extract the isosurface of the binary mask corresponding to each organ to convert the voxelized mask into a standard three-dimensional geometric shape, so as to obtain a triangular facet mesh model, the mesh is composed of a plurality of mesh nodes, each node has a unique three-dimensional coordinate, and a continuous closed tissue boundary is formed.

[0075] S15, obtaining the corresponding tissue organ properties according to the tissue organ type, and generating a finite element mesh in combination with the triangular facet mesh; the tissue organ properties include elasticity, tension and viscosity.

[0076] Illustratively, according to the identified tissue organ type, corresponding material mechanical property parameters are searched, including elastic modulus, shear modulus, viscous damping, tension coefficient, etc., which can be called based on the attribute table established based on the anatomical mechanical data table obtained from literature, histological research or actual measurement. Further, combined with the triangular facet mesh, it is converted into a volume mesh structure supporting finite element calculation, i.e. finite element mesh, through topology optimization, mesh refinement, etc. Each unit in the finite element mesh is assigned with corresponding material parameters to form a complete physical modeling system.

[0077] S16, model deformation and finite element analysis of physiological signals are performed according to the finite element mesh to obtain a three-dimensional digital physiological model.

[0078] Model deformation analysis and physiological signal simulation are performed based on the completed finite element mesh structure, i.e. finite element analysis (FEA). Specifically, standard boundary conditions and input parameters are loaded to the model, including contact force of surgical instruments, operation direction or internal pressure of organs, and stiffness matrix and mass matrix are called to solve the dynamic response process of the tissue, including displacement field, stress and strain field, tissue rupture interface, etc.; further, combined with the dynamic response of the model, a physiological dynamics model at the tissue level is constructed, and relevant physiological curves are output, including local stress-strain relationship, cutting impedance change, bleeding rate, tissue rebound rate, internal pressure change, etc. The finally formed three-dimensional digital physiological model not only contains geometric structure and apparent texture, but also integrates physical properties and physiological behaviors that can be driven to interact and respond.

[0079] In one embodiment, model deformation and finite element analysis of physiological signals are performed according to the finite element mesh to obtain a three-dimensional digital physiological model, including:

[0080] S21, stiffness matrix and mass matrix are obtained according to the finite element mesh.

[0081] The stiffness matrix and mass matrix of the target tissue are calculated according to the finite element mesh structure. The finite element mesh is composed of multiple nodes and elements, and each element defines its mechanical response relationship according to material properties and geometric shape. Specifically, using the standard finite element assembly process, the local stiffness matrix K (e) and mass matrix M (e) of each element are combined into the global system stiffness matrix K and mass matrix M according to the node index. The stiffness matrix represents the force required for unit displacement based on material properties such as elastic modulus, reflecting the anti-deformation ability of the tissue; the mass matrix represents the inertial response of the system to acceleration.

[0082] S22, deformation mode is obtained according to the stiffness matrix and mass matrix.

[0083] Based on the constructed K and M, the generalized eigenvalue Kφ = λMφ is solved to obtain the deformation mode of the tissue structure. Specifically, the eigenvalue λ obtained by modal analysis i The input corresponding to the model deformation is the eigenvector φ i which represents the deformation mode of the model under the input. By screening the first few low-order modes φ1, φ2, φ3, the response trend of the target tissue under typical loads, including cutting, pulling, compression, and other operations, is reconstructed. Since the modes have orthogonality and additivity, a real-time deformation model can be quickly constructed based on the linear superposition of multiple modes, improving the simulation performance.

[0084] S23, based on the Windkessel model, the blood vessel parameter curve is calculated according to the finite element grid; the blood vessel parameters include blood pressure and flow.

[0085] Illustratively, the Windkessel (elastic cavity model) model is introduced to simulate the dynamic characteristics of the local blood vessel. The Windkessel model is a widely used hemodynamic model, especially suitable for modeling the pressure-flow relationship of small blood vessel segments. According to the blood vessel region marked in the finite element grid, the volume feature and pipe diameter information in its node range are extracted, and combined with physiological parameters such as material impedance, blood viscosity, and cardiac output, a three-parameter Windkessel equivalent model is constructed, which can solve the pressure change curve of the target blood vessel region in unit time, i.e. blood pressure, and the flow velocity curve, i.e. flow, to simulate the blood flow response behavior caused by blood vessel compression, cutting, ligation, and other operations during the operation.

[0086] S24, map the blood vessel parameter curve to the stiffness matrix to obtain the bleeding rate function of local rupture.

[0087] Further, the blood vessel parameter curve is mapped to the corresponding stiffness matrix region to realize dynamic modeling of local bleeding simulation. Specifically, in the blood vessel node region, according to the blood pressure peak value, flow velocity gradient, and blood vessel wall shear stress, the local rupture risk factor is calculated, and when the risk factor exceeds the set threshold, it is determined that the blood vessel at this place has a rupture, and a local bleeding rate function is constructed. The input variables of the function are the blood pressure curve and the flow velocity curve, and the output is the bleeding amount per unit time, the bleeding speed, and the spatter distribution characteristics of the blood particles in space. This function is used to drive the visual particle system in the simulation scene in real time to achieve dynamic bleeding effects, including blood color change, flow velocity simulation, and collision feedback, thereby enhancing the reproduction ability of intraoperative emergencies.

[0088] S25, fuse the deformation mode, the blood vessel parameter curve, and the bleeding rate function to obtain a three-dimensional digital physiological model.

[0089] The aforementioned tissue deformation modalities, blood vessel parameter curves, and bleeding rate functions are fused and modeled to finally construct a three-dimensional digital physiological model with anatomical structure, mechanical response, and physiological feedback capability. Specifically, different types of data are synchronously scheduled and spatially mapped, the modal deformation is applied to the full model grid, the blood vessel nodes in the corresponding region are additionally subjected to pulsatile deformation and dynamic bleeding driven by blood pressure, and the non-vascular region remains static response or locally loads mechanical response according to instrument contact. The finally output three-dimensional digital physiological model can not only respond to user operation deformation in real time, but also generate visual and sensory feedback of the intraoperative state based on blood flow simulation.

[0090] In one of the embodiments, as shown in Figure 2 According to the teaching task configuration file, the corresponding three-dimensional digital physiological model is called, and special effect rendering is performed according to the clinical scene to obtain a visual virtual simulation experiment scene, including:

[0091] S201, material rendering is performed on the three-dimensional digital physiological model to obtain a simulation model.

[0092] Material rendering is performed on the three-dimensional digital physiological model to obtain a simulation model with real visual characteristics. Illustratively, material rendering not only includes basic color and texture mapping, but also includes fine setting of parameters such as reflectivity, transparency, light response, roughness, etc., to ensure that different tissues have identifiable optical characteristics in vision. Illustratively, a shader based on a PBR (Physically Based Rendering) physical rendering process is used to maintain uniform visual response characteristics under different light angles and interaction states.

[0093] S202, according to the local rupture position corresponding to the clinical scene, particle emission rendering is performed on the simulation model using the bleeding rate function to obtain a dynamic model.

[0094] Illustratively, according to the pre-set clinical scene in the teaching task configuration file, the spatial position information of the local rupture region is extracted, and particle emission rendering is performed based on the bleeding rate function to obtain a dynamic response model. Specifically, in the blood vessel distribution region or lesion edge of the simulation model, the emission frequency, direction, speed, and concentration of blood particles are dynamically calculated according to the current interaction state and input parameters of the bleeding rate function. Rendering is completed by using a particle system (Particle System) or a VFX Graph (Visual Effect Graph) parallel particle engine of Unity (a visualization engine tool) to complete visual simulation, supporting combination expression of different fluid behaviors such as splashing, flowing, solidifying, and atomization.

[0095] S203, map the dynamic model into a preset environment to obtain a visual virtual simulation experiment scene.

[0096] The dynamic model that completes the particle special effect rendering is mapped and deployed into a preset teaching simulation environment to generate a visual virtual simulation experiment scene. The environment can include a standard operating room layout or a specific scene setting, and corresponding background textures, lighting schemes, view angle control parameters, and interaction point definitions are loaded based on a preset scene template. By uniformly registering and aligning the tissue model and the environment resources in coordinates, the spatial scale, direction, and interaction nodes of the model in the simulation space are ensured to be consistent, and subsequent operation behaviors are ensured to occur at the correct positions.

[0097] In one of the embodiments, in response to the obtained user action data, the three-dimensional digital physiological model is deformed in real time, and feedback parameters are generated, including:

[0098] S31, based on the Unity engine, obtaining collision interaction events between the user and the three-dimensional digital physiological model according to the user action data.

[0099] The system establishes a spatial correlation between the interactive body and the target model based on the Unity engine, and listens to and captures collision interaction events between the user controller and the three-dimensional digital physiological model in real time. Specifically, the grid nodes or surface mapping areas of the three-dimensional digital physiological model are given physical collision properties, and the interactive body is bound with a space bounding box and a collision body. When the two contact, a collision event callback is triggered to identify the organ area corresponding to the current interaction part, the contact point coordinates, the collision depth, and other information.

[0100] S32, extracting time sequence-based penetration depth, position data, and attitude data of the user action data.

[0101] Illustratively, the user action data is time sequence-decomposed and parameter-extracted, and the penetration depth, interaction position data, and instrument attitude are extracted. The penetration depth represents the distance of the interactive body along the normal into the surface of the tissue model, and is a variable that determines the model deformation amplitude and the tissue response degree. The position data describes the real-time position of the interaction point in the world coordinate system or the local organ coordinate system, and is used to locate the interaction range and node index. The attitude data is used to determine the operator's control direction of the instrument, and then determine whether there is a rotation and traction composite operation behavior.

[0102] S33, inputting the penetration depth, position data, and attitude data into a deformation mode to obtain linear superimposed deformation data of the corresponding three-dimensional digital physiological model.

[0103] The penetration depth, position data and pose data are input into the established model deformation modal, and the deformation state of the current model under the force condition is calculated by using linear superposition. Specifically, based on a plurality of deformation modal equations φ i , the activation strength of each modal is solved according to the normalized value of the penetration depth and the pose guide direction, and each order modal deformation is superimposed to obtain the complete tissue deformation result.

[0104] S34, according to the penetration depth and the corresponding time sequence, the mechanical feedback data of the three-dimensional digital physiological model is calculated.

[0105] Further, according to the penetration depth and the interaction timestamp, the current acting force is calculated to construct the mechanical feedback data. Specifically, when the interactive body penetrates the tissue surface at a certain speed, the local deformation energy of the surface node is calculated, and the corresponding reaction force size and direction are output accordingly; if there is a quick pull-out or rotation traction in the operation process, a dynamic damping term and a shear stress term will also be superimposed to reflect the real touch difference. The mechanical feedback data usually includes the action force vector value, the action point position, the tissue recovery speed and the impedance index, which can be used in real time for subsequent haptic driving.

[0106] S35, generating feedback parameters according to the mechanical feedback data; the feedback parameters include vibration parameters.

[0107] According to the mechanical feedback data, feedback parameters are generated for physical feedback devices to execute, which include but are not limited to vibration intensity, frequency, pulse mode, duration, etc. The generation logic of the feedback parameters usually establishes a linear or nonlinear correspondence between the force amplitude and the vibration frequency based on a preset mapping model. For example, a small contact force corresponds to a low-frequency light touch feedback, and a fast puncture or injury triggers a high-frequency short pulse feedback. For devices supporting multi-channel haptic output, different touch mode parameter tables can also be loaded based on the tissue type. Optionally, the feedback parameters are transmitted to the force feedback device through Bluetooth, serial port or middleware driving module.

[0108] In one embodiment, as shown in Figure 3 , the user action data is compared with the preset standard expert action data, and a teaching evaluation result is generated based on the evaluation standard, including:

[0109] S301, obtaining corresponding standard expert action data according to the collision interaction event; the standard expert action data includes expert template trajectory.

[0110] Illustratively, according to the collision interaction event between the user and the three-dimensional model, the organ region and the surgical step involved in the operation are determined, and then the standard expert action data corresponding to the step is called. Illustratively, the standard expert action data is a data set formed by collecting the operation of a senior clinical expert under the same teaching task and simulation scene, and the data structure includes the operation position trajectory, posture change, operation rhythm and timestamp information of the expert.

[0111] S302, obtaining a student experiment trajectory vector according to the position data and posture data based on time sequence.

[0112] Illustratively, according to the real-time collected user action data, the position change sequence and the posture change sequence in the whole operation process are extracted, and the student's experiment trajectory vector is constructed in the order of timestamp. The trajectory vector is represented in a discrete sequence, which records the spatial coordinates and posture information of the current interactive instrument at each time point, thereby forming an operation path continuously evolving in space-time.

[0113] S303, comparing the student experiment trajectory vector with the expert template trajectory based on a preset speed tolerance error window of dynamic time rule, to obtain a trajectory error.

[0114] The trajectory comparison mechanism based on the dynamic time warping (DTW) algorithm is introduced, combined with the common operation rhythm change characteristics in the operation, the user trajectory and the expert template trajectory are compared frame by frame under the preset speed tolerance error window, and the trajectory error is calculated. The DTW algorithm can tolerate nonlinear changes in execution speed, and is suitable for the demand of surgical operation trajectory comparison. On the basis of DTW, a speed window control is introduced, that is, the maximum offset frame number at both ends of the time axis is limited, so as to avoid evaluation deviation caused by extreme alignment. The error calculation dimensions can include spatial Euclidean distance error, posture angle error, path offset rate, overall similarity score, etc., wherein the spatial error calculation can use three-dimensional coordinate difference vector norm, and the posture error can use the angle formula corresponding to the quaternion dot product.

[0115] S304, obtaining a teaching evaluation result according to the trajectory error based on the evaluation standard; the teaching evaluation result includes a precision score.

[0116] Illustratively, the obtained trajectory error is taken as input, combined with the preset evaluation standard in the teaching task configuration file, and the final teaching evaluation result is generated. The evaluation standard is usually set by the teaching designer or teaching research expert according to the teaching goal, and the content can include error threshold division, score function design, weighting rule setting, etc. Illustratively, a score mechanism based on Sigmoid or Piecewise function is adopted, which maps the trajectory error to a precision score of 0-100, and marks the corresponding level.

[0117] In one embodiment, the trajectory error is obtained by the following formula:

[0118]

[0119] wherein RMSE is the trajectory error; T is the total length of the trajectory; post[t] is the student experimental trajectory vector of the t-th frame; post[t] is the expert template trajectory of the t-th frame. expert

[0120] It should be understood that, although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0121] Based on the same inventive concept, the embodiments of the present application also provide a clinical surgical experiment virtual simulation teaching device for implementing the above-mentioned clinical surgical experiment virtual simulation teaching method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more clinical surgical experiment virtual simulation teaching device embodiments provided below can refer to the limitations of the clinical surgical experiment virtual simulation teaching method described above, which will not be repeated here.

[0122] In one exemplary embodiment, as shown in Figure 4 a clinical surgical experiment virtual simulation teaching device is provided, comprising:

[0123] The teaching experiment task configuration module 401 is configured to respond to the acquired teaching task configuration file; the teaching task configuration file includes a clinical scene and an evaluation standard;

[0124] The simulation module 402 is configured to call a corresponding three-dimensional digital physiological model according to the teaching task configuration file, and perform special effect rendering according to the clinical scene to obtain a visual virtual simulation experiment scene;

[0125] The interaction module 403 is configured to respond to the acquired user action data, deform the three-dimensional digital physiological model in real time, and generate feedback parameters;

[0126] ​The teaching evaluation module 404 is configured to compare the user action data with preset standard expert action data, and generate a teaching evaluation result based on an evaluation standard.

[0127] In one of the embodiments, the system further comprises:

[0128] The data acquisition module is configured to acquire a set of medical anatomical images.

[0129] The data extraction module is configured to perform volume data extraction on the medical anatomical images to obtain a volume data set.

[0130] The target extraction module is configured to classify tissue organs of the medical anatomical images by using a segmentation algorithm, and combine the volume data to generate tissue organ types, corresponding binary masks and confidence levels.

[0131] The model construction module is configured to perform isosurface extraction on the binary masks to obtain a triangular facet mesh.

[0132] The model construction module is further configured to obtain corresponding tissue organ attributes according to the tissue organ types, and combine the triangular facet mesh to generate a finite element mesh; the tissue organ attributes include elasticity, tension and viscosity.

[0133] The model construction module is further configured to perform model deformation and finite element analysis of physiological signals according to the finite element mesh to obtain a three-dimensional digital physiological model.

[0134] In one of the embodiments, the system further comprises:

[0135] The finite element analysis module is configured to obtain a stiffness matrix and a mass matrix according to the finite element mesh.

[0136] The deformation response module is configured to obtain a deformation mode according to the stiffness matrix and the mass matrix.

[0137] The physiological simulation module is configured to calculate a blood vessel parameter curve according to the finite element mesh based on a Windkessel model; the blood vessel parameters include blood pressure and flow rate.

[0138] The physiological simulation module is further configured to map the blood vessel parameter curve into the stiffness matrix to obtain a local rupture bleeding rate function.

[0139] The model construction module is further configured to fuse the deformation mode, the blood vessel parameter curve and the bleeding rate function to obtain the three-dimensional digital physiological model.

[0140] In one of the embodiments, the system further comprises:

[0141] The material rendering module is configured to perform material rendering on the three-dimensional digital physiological model to obtain a simulation model.

[0142] a dynamic rendering module, configured to perform particle emission rendering on the simulation model according to the local rupture position corresponding to the clinical scene by using a bleeding rate function, to obtain a dynamic model;

[0143] a scene mapping module, configured to map the dynamic model into a preset environment to obtain a visual virtual simulation experiment scene.

[0144] In one of the embodiments, the system further comprises:

[0145] an interaction detection module, configured to obtain a collision interaction event between the user and the three-dimensional digital physiological model according to user action data based on a Unity engine;

[0146] an action detection module, configured to extract time-series-based penetration depth, position data and posture data of the user action data;

[0147] the deformation response module is further configured to input the penetration depth, position data and posture data into a deformation mode to obtain linear superimposed deformation data of the corresponding three-dimensional digital physiological model;

[0148] a mechanical feedback module, configured to calculate mechanical feedback data of the three-dimensional digital physiological model according to the penetration depth and the corresponding time sequence;

[0149] a feedback parameter module, configured to generate feedback parameters according to the mechanical feedback data; the feedback parameters include vibration parameters.

[0150] In one of the embodiments, the system further comprises:

[0151] a template library, configured to obtain corresponding standard expert action data according to the collision interaction event; the standard expert action data includes an expert template trajectory;

[0152] a trajectory detection module, configured to obtain a trainee experiment trajectory vector according to the time-series-based position data and the posture data;

[0153] a comparison module, configured to compare the trainee experiment trajectory vector with the expert template trajectory based on a preset speed tolerance error window of a dynamic time warping rule to obtain a trajectory error;

[0154] the teaching evaluation module 404 is further configured to obtain a teaching evaluation result according to the trajectory error.

[0155] In one embodiment, a computer device is provided, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0156] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above-mentioned method embodiments.

[0157] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can be seen from the part of the method embodiment. The device embodiment described above is only schematic, wherein the components shown as separate components can or can not be physically separate, and the components shown as a unit can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure. Those skilled in the art can understand and implement without creative labor.

[0158] The above-described embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that, for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application.

Claims

1. A virtual simulation teaching method for clinical surgical experiments, characterized in that: The method comprises: In response to the acquired teaching task configuration file, the teaching task configuration file includes a clinical scenario and evaluation criteria; Calling the corresponding three-dimensional digital physiological model according to the teaching task configuration file, and performing special effects rendering according to the clinical scene to obtain a visual virtual simulation experiment scene; In response to the acquired user motion data, the three-dimensional digital physiological model is deformed in real time and feedback parameters are generated; the feedback parameters are used to instruct the interactive control to operate according to the feedback parameters; The user action data is compared with preset standard expert action data, and a teaching evaluation result is generated based on the evaluation standard.

2. The method according to claim 1, characterized in that The three-dimensional digital physiological model is constructed by the following method: Acquire a medical anatomical image set; the medical anatomical image set includes medical anatomical images corresponding to a plurality of clinical scenarios; Extracting volume data from the medical anatomical image to obtain a volume data set; Using a segmentation algorithm to classify tissues and organs in the medical anatomical image, and combining the volume data to generate tissue and organ types and corresponding binary masks and confidence levels; Performing isosurface extraction on the binary mask to obtain a triangular face mesh; the triangular face mesh includes a plurality of mesh nodes and corresponding node coordinates; Obtaining corresponding tissue and organ properties according to the tissue and organ type, and generating a finite element mesh by combining the triangular patch mesh; the tissue and organ properties include elasticity, tension, and viscosity; Finite element analysis of model deformation and physiological signals is performed based on the finite element grid to obtain the three-dimensional digital physiological model.

3. The method according to claim 2, characterized in that The step of performing finite element analysis on model deformation and physiological signals according to the finite element grid to obtain the three-dimensional digital physiological model includes: Obtaining a stiffness matrix and a mass matrix based on the finite element mesh; Obtaining a deformation mode according to the stiffness matrix and the mass matrix; Based on the Windkessel model, a vascular parameter curve is obtained according to the finite element grid calculation; the vascular parameters include blood pressure and flow; Mapping the blood vessel parameter curve into the stiffness matrix to obtain a bleeding rate function of a local rupture; The deformation mode, the blood vessel parameter curve and the bleeding rate function are fused to obtain the three-dimensional digital physiological model.

4. The method according to claim 3, characterized in that The method of calling the corresponding three-dimensional digital physiological model according to the teaching task configuration file and performing special effects rendering according to the clinical scene to obtain a visual virtual simulation experiment scene includes: Performing material rendering on the three-dimensional digital physiological model to obtain a simulation model; According to the local rupture position corresponding to the clinical scenario, the simulation model is rendered by particle emission using the bleeding rate function to obtain a dynamic model; The dynamic model is mapped to a preset environment to obtain the visual virtual simulation experiment scene.

5. The method according to claim 3, characterized in that The step of deforming the three-dimensional digital physiological model in real time and generating feedback parameters in response to the acquired user motion data includes: Based on the Unity engine, obtaining a collision interaction event between the user and the three-dimensional digital physiological model according to the user action data; Extracting time-series-based penetration depth, position data, and posture data of the user motion data; Inputting the penetration depth, position data and posture data into the deformation modality to obtain linear superposition deformation data corresponding to the three-dimensional digital physiological model; Calculating mechanical feedback data of a three-dimensional digital physiological model according to the penetration depth and the corresponding time sequence; Feedback parameters are generated according to the mechanical feedback data; the feedback parameters include vibration parameters.

6. The method according to claim 5, characterized in that The comparing the user motion data with the preset standard expert motion data and generating a teaching evaluation result based on the evaluation criteria includes: Obtaining the corresponding standard expert action data according to the collision interaction event; the standard expert action data includes an expert template trajectory; Obtaining a student's experimental trajectory vector according to the time-series-based position data and the posture data; Based on a preset speed tolerance window of a dynamic time rule, the student's experimental trajectory vector is compared with the expert template trajectory to obtain a trajectory error; Based on the evaluation criteria, the teaching evaluation result is obtained according to the trajectory error; the teaching evaluation result includes an accuracy score.

7. The method according to claim 6, characterized in that: The trajectory error is obtained by the following formula: Among them, RMSE is the trajectory error; T is the total duration of the trajectory; post[t] is the student experimental trajectory vector of the tth frame; pos expert [t] is the expert template trajectory of the t-th frame.

8. A virtual simulation teaching device for clinical surgical experiments, characterized in that: The device comprises: A teaching experiment task configuration module, configured to respond to the acquired teaching task configuration file; the teaching task configuration file includes a clinical scenario and evaluation criteria; A simulation module is used to call the corresponding three-dimensional digital physiological model according to the teaching task configuration file, and perform special effects rendering according to the clinical scene to obtain a visual virtual simulation experiment scene; an interaction module, configured to deform the three-dimensional digital physiological model in real time in response to the acquired user motion data and generate feedback parameters; The teaching evaluation module is used to compare the user action data with preset standard expert action data and generate a teaching evaluation result based on the evaluation standard.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images