Thoracic surgery teaching assisting method and system

Abstract

The invention provides a thoracic surgery teaching assistance method and system, and the method comprises the steps: constructing a thoracic surgery virtual operation interaction environment based on a digital twin technology, and endowing a virtual thoracic cavity tissue model with initial biomechanical parameters; an operation instruction data sequence of an operator for the virtual surgical instrument is collected in real time through the man-machine interaction interface; based on the operation instruction data sequence, calculating an operation disturbance characteristic index representing the motion dispersion degree of an operator in real time by using a sliding time window algorithm; establishing a dynamic evolution mapping model of an operation disturbance characteristic index and a biomechanical parameter, dynamically correcting an initial biomechanical parameter of the virtual tissue according to the index calculated in real time, and when the index exceeds the standard, controlling the parameter to deviate to an easily damaged state so as to simulate histopathologic change caused by rough operation; and performing physical collision response calculation and three-dimensional picture rendering according to the evolved model. According to the invention, dynamic coupling of operation behaviors and organization characteristics is realized, and the physical reality sense of teaching is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical virtual simulation and computer-aided teaching, in particular to a thoracic surgery teaching auxiliary method and system. BACKGROUND

[0002] Thoracic surgery involves important organs such as lungs, trachea, mediastinum and large blood vessels, and the anatomical structure is complex and has many variations, which requires high stability, delicacy of the surgeon's operation and cognition of the tissue characteristics. With the rapid development of computer simulation technology and virtual reality technology, virtual surgery system has gradually become an important auxiliary tool for medical education and clinical skill training. By constructing a digital virtual dissection scene, trainees can repeatedly practice key surgical actions such as cutting, suturing and separating under thoracoscopy in a risk-free environment, which has a positive significance for shortening the learning curve and reducing the risk of clinical practice.

[0003] In the existing virtual surgery teaching training system, the design of the virtual human soft tissue model mainly focuses on the fidelity of the geometric appearance and the basic deformation simulation. Usually, the biomechanical parameters of the virtual organ such as elastic modulus, Poisson's ratio and shear stress threshold are set as fixed values in the system initialization stage, and are regarded as constant physical properties throughout the training process. The system mainly determines the contact state and surgical effect according to the geometric position relationship between the end of the surgical instrument and the tissue surface, and pays less attention to the real-time influence of the quality of the operation process itself on the physical properties of the tissue.

[0004] However, in real clinical surgery, biological tissues have complex biomechanical response characteristics and pathological and physiological evolution mechanisms. The behavior pattern of the surgical operator will directly change the physical state of the tissue, for example, long-time instrument extrusion, repeated invalid traction or micro-vibration of the hand due to tension, which will all cause phenomena such as congestion, edema, decreased compliance or increased brittleness of the tissue. The existing virtual simulation technology lacks the dynamic coupling mechanism between the operation behavior and the tissue characteristics, and cannot real-time feedback the pathological changes of the tissue according to the roughness of the trainee's operation. This makes the trainee often pay too much attention to the smoothness of the surgical procedure in the simulation training, and ignore the implementation of the minimally invasive operation concept, making it difficult to develop the consciousness of protecting the tissue in the virtual environment, and limiting the application value of the simulation teaching in cultivating high-level clinical operation thinking. SUMMARY

[0005] The present application aims to provide a thoracic surgery teaching auxiliary method and system to solve the problems pointed out in the background.

[0006] In a first aspect, the present application provides a thoracic surgery teaching auxiliary method, which comprises the following steps: Construct a thoracic surgery virtual surgery interactive environment based on digital twin technology, the virtual surgery interactive environment includes a virtual thoracic tissue model endowed with initial biomechanical parameters, the initial biomechanical parameters at least define the elastic modulus coefficient of the virtual tissue and the tissue rupture threshold; Real-time collection of operator operation instruction data sequence for virtual surgery instruments through human-computer interaction interface, the operation instruction data sequence contains a set of spatial pose vectors of the virtual surgery instruments on the time axis; Based on the operation instruction data sequence, the operation disturbance characteristic index of the operator is calculated in real time by using the preset sliding time window algorithm, the operation disturbance characteristic index is used to represent the action dispersion degree of the operator in unit time; Establish a dynamic evolution mapping model of operation disturbance characteristic index and biomechanical parameters, according to the real-time calculated operation disturbance characteristic index, the dynamic evolution mapping model is used to dynamically correct the initial biomechanical parameters of the virtual thoracic tissue model, and an evolved virtual thoracic tissue model is generated; When the operation disturbance characteristic index exceeds the preset safety interval, the initial biomechanical parameters are offset to the vulnerable state to simulate the change of tissue pathological and physiological characteristics caused by rough operation in surgery; According to the evolved virtual thoracic tissue model, the subsequent operation instruction data sequence of the operator is calculated and three-dimensional picture rendering by using the graphic rendering engine.

[0007] Optionally, the step of constructing a thoracic surgery virtual surgery interactive environment based on digital twin technology comprises: Obtain the tomographic image data of the patient's chest, and generate a static grid model containing lung lobes, bronchi and pulmonary vessels by using three-dimensional reconstruction technology; Mechanical property assignment is performed on the vertices of the static grid model, and the mechanical property assignment is set based on the physical measurement data of healthy human thoracic tissue; Configure virtual cameras and virtual light sources in the virtual surgery interactive environment, and establish a rendering pipeline for simulating thoracoscope field of view.

[0008] Optionally, the step of real-time collection of operator operation instruction data sequence for virtual surgery instruments through human-computer interaction interface comprises: Read the six-degree-of-freedom sensor signal output by the human-computer interaction interface at a preset high-frequency sampling rate; The six-degree-of-freedom sensor signal is processed by Kalman filtering to filter out environmental noise interference, and the coordinate position and attitude angle of the virtual surgery instrument end are calculated to form the spatial pose vector set.

[0009] Optionally, the step of performing physical collision response calculation and three-dimensional picture rendering on the subsequent operation instruction data sequence of the operator by using the graphic rendering engine comprises: constructing a hierarchical bounding volume index structure of the virtual thoracic tissue model; in each rendering frame period, traversing the hierarchical bounding volume index structure by using the set of spatial pose vectors to detect whether geometric contact occurs between the virtual surgical instrument and the virtual thoracic tissue model; if geometric contact is detected, calculating a grid deformation variable of a contact area according to the current elastic modulus coefficient, and updating image data in a display buffer.

[0010] Optionally, the method further comprises a basic teaching evaluation step: recording total time consumption of the operator for completing a preset surgical action and cumulative movement distance of the virtual surgical instrument; comparing the total time consumption and the cumulative movement distance with standard expert data to calculate a basic operation score; generating a teaching analysis report containing the basic operation score.

[0011] Optionally, the step of calculating the operation disturbance characteristic index of the operator in real time based on the operation instruction data sequence by using a preset time series data analysis algorithm comprises: extracting a kinematic feature sequence in the set of spatial pose vectors; calculating a dispersion statistical indicator of the kinematic feature sequence within a statistical window; detecting a trajectory smoothness indicator of the set of spatial pose vectors within the statistical window; weighting and fusing the dispersion statistical indicator and the trajectory smoothness indicator to obtain the operation disturbance characteristic index representing a degree of hand tremor and a degree of operation hesitation of the operator.

[0012] Optionally, the step of establishing a dynamic evolution mapping model of the operation disturbance characteristic index and the biomechanical parameter and dynamically correcting the initial biomechanical parameter comprises: defining a cumulative damage influence factor, the cumulative damage influence factor being nonlinearly increased with an increase of the operation disturbance characteristic index; configuring a tissue viscoelastic recovery logic, when it is monitored that the operation disturbance characteristic index continuously stays within a preset time period, controlling the cumulative damage influence factor to be inversely rolled back according to a preset time decay function to simulate an elastic self-repairing characteristic of biological tissue; performing decay operation on the elastic modulus coefficient by using the cumulative damage influence factor, so that the virtual thoracic tissue model exhibits a tissue relaxation characteristic; The cumulative damage influence factor is used for nonlinear attenuation operation on the tissue rupture threshold, so that the virtual thoracic tissue model is more likely to trigger a rupture logic when subjected to the same external force.

[0013] Optionally, the method further comprises a visual feedback step based on parameter evolution: Real-time monitoring of the change amplitude of the tissue rupture threshold; When the tissue rupture threshold is reduced to below a preset dangerous value, automatically adjusting the surface texture map of the virtual thoracic tissue model to superimpose a visual special effect of tissue congestion or edema; A dynamic heat map is generated around the tip of the virtual surgical instrument, and the color depth of the dynamic heat map is positively correlated with the current cumulative damage influence factor.

[0014] Optionally, the method further comprises a haptic feedback control step based on the evolved model: According to the biomechanical parameters of the evolved virtual thoracic tissue model, the reaction force data when the virtual surgical instrument cuts or pulls the tissue is calculated; The reaction force data is converted into a force feedback control signal and sent to the human-computer interaction interface; When the biomechanical parameters are in a vulnerable state, control the force feedback control signal to produce high-frequency micro-vibration to simulate the tactile texture difference of the diseased tissue.

[0015] In a second aspect, the present application provides a thoracic surgery teaching auxiliary system, the system comprises: A virtual environment construction module configured to construct a virtual surgical interactive environment containing a virtual thoracic tissue model endowed with initial biomechanical parameters; A data acquisition module configured to acquire a sequence of operation instruction data of an operator for a virtual surgical instrument in real time through a human-computer interaction interface; A feature calculation module configured to calculate an operation disturbance feature index of the operator in real time based on the sequence of operation instruction data; A parameter evolution control module configured to dynamically adjust the biomechanical parameters of the virtual thoracic tissue model according to the operation disturbance feature index to generate an evolved virtual thoracic tissue model; A rendering interaction module configured to perform physical collision response calculation and three-dimensional picture rendering according to the evolved virtual thoracic tissue model using a graphics rendering engine; The system further comprises a processor and a memory storing a computer program, and the processor implements the method steps of any one of the first aspect when executing the computer program.

[0016] The present application has the following beneficial effects: The present application realizes a thoracic surgery teaching environment with adaptive evolution capability by constructing an operation disturbance characteristic index and a dynamic evolution mapping model of biomechanical parameters. The system uses a sliding time window algorithm to quantitatively analyze the acceleration variance and motion direction mutation frequency of the operator's hand in real time, converts the fine physiological tremor or psychological hesitation into a calculable disturbance index, and dynamically adjusts the physical properties of the virtual tissue accordingly. This mechanism enables the virtual lung tissue or blood vessel to simulate the real pathological processes such as tissue relaxation, edema or rupture threshold reduction caused by rough operation, forcing the trainee to maintain the stability and force control of the operation at all times while focusing on the operation steps, thereby effectively cultivating the fine operation habits in line with the clinical standards.

[0017] The present application also establishes a multi-modal feedback system including vision and touch, enhancing the immersion and guidance of teaching training. On the visual level, the system automatically adjusts the texture mapping of the tissue surface according to the cumulative damage influence factor, presents the visual special effects of hyperemia or edema, and intuitively identifies the high-risk areas through a dynamic heat map; on the tactile level, the mechanical texture in line with the evolved biomechanical parameters is fed back through the human-computer interaction interface, and a micro-vibration signal is triggered when the tissue is in a vulnerable state. This multi-modal feedback method can correct the trainee's wrong actions in real time during the operation, helping them establish correct muscle memory and risk perception ability.

[0018] The present application uses Kalman filtering technology and high-frequency sampling strategy to ensure the accuracy and real-time performance from motion capture to physical calculation. The system can effectively filter out environmental noise and equipment mechanical interference, and accurately capture the low-frequency tremor signal of physiological significance of the operator's hand. By incorporating stability data during the operation into the teaching evaluation system, the present application solves the problem of single evaluation dimension of traditional simulators, and provides a digital training platform with deep physical interaction capability for skill assessment and growth of thoracic surgeons.

[0019] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and the appended drawings.

[0020] The technical solutions of the present application will be further described in detail below with the help of the accompanying drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0021] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, together with the embodiments of the present application, to explain the present application, and do not constitute a limitation on the present application. In the drawings: Figure 1A schematic diagram of a thoracic surgery teaching auxiliary method in an embodiment of the present application; Figure 2 A schematic diagram of a thoracic surgery teaching auxiliary system in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The preferred embodiments of the present application will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to explain and illustrate the present application, and are not used to limit the present application.

[0023] In a conventional virtual surgery system, a human tissue model is usually set as a static object with constant physical properties (such as a fixed elastic modulus, a fixed shear stress threshold), and cannot truly feedback the pathophysiological changes (such as hyperemia, edema, and increased fragility) caused by non-standard operations (such as hand microtremor and rough pulling) of the operator due to psychological fluctuations (such as nervousness and hesitation) during the operation on the biological tissue.

[0024] To this end, as shown in Figure 1 The present application discloses a thoracic surgery teaching auxiliary method, which is executed based on a hardware environment composed of a high-performance graphics workstation, a force feedback human-computer interaction device, and a high-resolution display terminal. The method is logically divided into core stages of environment construction, data acquisition, feature calculation, parameter evolution, rendering interaction, and teaching evaluation, and real-time data exchange is performed between the stages through shared memory or a high-speed data bus. The method specifically includes the following steps: Step S11: Construct a thoracic surgery virtual surgery interactive environment based on digital twinning technology, the virtual surgery interactive environment including a virtual thoracic tissue model endowed with initial biomechanical parameters, the initial biomechanical parameters at least defining an elastic modulus coefficient and a tissue rupture threshold of the virtual tissue.

[0025] In the present embodiment, constructing a high-fidelity virtual surgery interactive environment is a basic physical platform for realizing effective teaching and accurate evaluation. The virtual surgery interactive environment performs three-dimensional scene reproduction in vision, and also performs deep digital mapping of physical properties and biological characteristics. The virtual thoracic tissue model has an entity model of internal physical topological structure and biomechanical response mechanism. It should be noted that the digital twinning in the present specification specifically refers to a hybrid simulation mechanism combining a static anatomical structure reconstructed based on real anatomical image data of a patient and dynamic mapping of real-time operation behavior data of an operator, rather than real-time physiological signal synchronization with a patient entity on an operating table.

[0026] Specifically, the step of constructing a thoracic surgery virtual operation interactive environment based on digital twinning technology first carries out acquisition and processing of medical image data. The system imports high-resolution tomographic image data (CT or MRI) of the chest of a real patient through a medical digital imaging and communication interface. In order to ensure the degree of restoration of microanatomy, the thickness of the image layer is preferably controlled between 0.5 mm and 1.0 mm to capture key structures such as subsegmental bronchi, microvascular branches, etc. Using three-dimensional reconstruction technology, the system first carries out noise reduction preprocessing of the original image data, and adopts an anisotropic diffusion filtering algorithm to remove imaging noise while preserving tissue edge gradients. Subsequently, a U-Net semantic segmentation network based on deep learning is applied in combination with a traditional region growing algorithm to accurately segment the lung lobes, bronchial tree (i.e. trachea to subsegmental bronchi), pulmonary arterial and venous vascular network, and mediastinal lymph nodes, etc. After obtaining the voxelized segmentation data, the surface isosurface of each tissue is extracted using a marching cubes algorithm to generate a preliminary surface triangular mesh model.

[0027] Further, in order to support subsequent physical simulation calculations, especially to simulate the volume deformation and internal stress transmission of soft tissue, the embodiment adopts a tetrahedral mesh generation technology to fill the surface-closed triangular mesh into a volume mesh model composed of tetrahedral elements. Each node (Vertex) and element (Element) of the static mesh model is assigned a specific physical identifier to constitute the smallest unit of physical calculation.

[0028] On this basis, the vertices of the static mesh model are assigned mechanical properties. The mechanical property assignment is based on physical measurement data of the healthy human thoracic tissue. The system has a built-in database containing multiple types of thoracic tissue biomechanical parameters. For lung parenchymal tissue, considering its sponge-like porous structure and high compliance, the system assigns it a lower initial elastic modulus coefficient (Young's Modulus, for example, 1.0-3.0 kPa) and a Poisson's ratio close to 0.45 to simulate its incompressibility and good rebound characteristics. For lung blood vessels, considering that the vessel wall contains elastic fibers and smooth muscle, it is given the nonlinear hyperelastic material properties (such as Neo-Hookean or Mooney-Rivlin constitutive model parameters) and a higher initial tensile strength. For bronchial cartilage, a higher stiffness coefficient is set to reflect its supporting role. The initial biomechanical parameters also define the tissue rupture threshold, which is a comprehensive physical quantity, usually represented by the limit value of the maximum principal stress or Von Mises stress. For example, for a healthy lung vein wall, the initial rupture threshold can be set to withstand a shear force of 2.5 N or a tensile deformation of 150%. These parameters collectively define a healthy, standard virtual surgical object as a baseline state for subsequent dynamic evolution.

[0029] In addition, in order to create an immersive visual experience, a virtual camera and a virtual light source are configured in the virtual surgery interactive environment to establish a rendering pipeline for simulating the thoracoscope field of view. The intrinsic matrix (focal length, principal point coordinates) and distortion coefficient of the virtual camera are strictly set in accordance with the optical properties of a real thoracoscope (such as a 30-degree mirror), and post-processing filters are applied to simulate lens vignetting, depth of field, and color deviation. The virtual light source simulates the point light source characteristics of the endoscope front cold light source, and combines a physically-based rendering material system to calculate the reflection, refraction, and subsurface scattering of light on the wet and smooth organ surface, thereby realistically reproducing the dim, reflective, and layered visual environment inside the thoracic cavity.

[0030] Step S12: Real-time collection of operator operation instruction data sequence for virtual surgical instruments through human-computer interaction interface, the operation instruction data sequence containing a set of spatial pose vectors of the virtual surgical instruments on the time axis.

[0031] In this embodiment, the human-computer interaction interface usually adopts a master hand device with six degrees of freedom (6-DoF) position input and three degrees of freedom force feedback output, such as Geomagic Touch or Omega series devices.

[0032] Specifically, the step of acquiring the operator's operation command data sequence for virtual surgical instruments in real time through the human-computer interaction interface includes: reading the six-degree-of-freedom sensor signals output by the human-computer interaction interface at a preset high-frequency sampling rate. To capture extremely subtle physiological tremors or psychological shaking of the operator's hand, the system sets the sampling frequency to no less than 1000Hz (i.e., a time resolution of 1ms). This high-frequency sampling rate is much higher than the visual rendering frame rate (typically 60Hz), and its purpose is to meet the stability requirements of the tactile control loop and the needs of high-frequency motion feature analysis. Specifically, the system, according to the preset sampling frequency... ( Define the time step for discrete computation. Corresponding to a duration of ( The sliding time window, which contains the number of discrete sampling points. The calculation is as follows: Number of sampling points This will be used in subsequent calculations of acceleration variance. The statistical sample size is large enough to ensure that the statistical results have sufficient confidence.

[0033] The raw signals output by the sensor include displacement along the X, Y, and Z axes, as well as Euler angles (Yaw, Pitch, Roll) or quaternions of rotation around the three axes.

[0034] However, the original sensor signals inevitably contain mechanical noise from the mechanical transmission gaps of the equipment and high-frequency electromagnetic noise from the circuitry. To obtain clean motion data that truly reflects the operator's intentions, this embodiment performs Kalman filtering on the six-degree-of-freedom sensor signals. The Kalman filter establishes the system's state equation (describing the instrument's motion) and observation equation (describing the sensor's measurement process), uses the state estimate from the previous moment to predict the prior state at the current moment, and combines this with the current observation to correct the prior state, thereby obtaining the optimal posterior state estimate. By finely adjusting the process noise covariance matrix Q and the observation noise covariance matrix R, the system can effectively filter out high-frequency noise unrelated to hand movements while retaining physiologically significant low-frequency tremors (typically in the 8-12Hz range), thus calculating the precise coordinate position and posture angle of the virtual surgical instrument's end effector. In the specific configuration of this embodiment, to filter out the equipment's electromagnetic noise while retaining the operator's 8-12Hz physiological tremor characteristics of the hand, the process noise covariance matrix of the Kalman filter... Set the diagonal element value The diagonal matrix, observation noise covariance matrix Set the diagonal element value diagonal matrix. This parameter combination ensures high reliability of the system to the operator's small displacement, while smoothing the high-frequency mechanical jitter that is not physiological.

[0035] These processed position and pose data are arranged in time stamp order to form the spatial pose vector set where each vector contains equal components. Among them, represents the three-dimensional position coordinates of the virtual surgical instrument tip in the virtual environment world coordinate system; represents the quaternion component describing the pose of the instrument tip, which is used to avoid gimbal lock problems; represents the sampling time stamp when the data frame is collected. This operation instruction data sequence serves as the instruction source for driving the virtual instrument motion and is the core data basis for subsequent analysis of the operator's psychological state and technical level.

[0036] Step S13: Based on the operation instruction data sequence, a preset sliding time window algorithm is used to calculate the operation disturbance characteristic index of the operator in real time, which is used to represent the action dispersion degree of the operator in unit time.

[0037] In this embodiment, the purpose of this step is to quantitatively evaluate the operator's “soft skills” (psychological quality and hand stability). Traditional teaching systems often only focus on the operation result (correctness of operation steps), but ignore the quality in the operation process (stability of operation method). The present application proposes the operation disturbance characteristic index as an indicator for real-time quantitative evaluation of the action non-stationary of the operator due to tension, fatigue or lack of technical proficiency.

[0038] Specifically, the step of calculating the operation disturbance characteristic index of the operator based on the operation instruction data sequence using a preset sliding time window algorithm includes: First, extract the acceleration component sequence in the spatial pose vector set. Since the pose vector directly provides position information, the system uses the second-order central difference method (Central Difference Method) to perform differential operation on consecutive position coordinates, thereby obtaining the instantaneous linear acceleration and angular acceleration of the instrument tip. Acceleration is a direct physical quantity reflecting force and muscle control state, and it is more sensitive to small jitter than speed or position.

[0039] Further, a sliding time window algorithm is applied. The system sets a window with a time length of (e.g. 500ms), which slides forward with the input of real-time data stream. In each time step, the acceleration data sequence within the window range is intercepted.

[0040] Then, the variance value of the acceleration component sequence within the sliding time window is calculated. Variance is an index in statistics to measure the degree of data dispersion. If the operator is confident and calm, his actions are usually smooth and continuous, the acceleration changes continuously, and the variance value is small; on the contrary, if the operator is in a state of extreme tension, the hand will appear uncontrollable tremor, causing the acceleration to have a sharp high-frequency oscillation in a short time, and the variance value will significantly increase.

[0041] At the same time, the frequency of sudden change of motion direction of the set of spatial pose vectors within the sliding time window is detected. This is mainly used to evaluate the degree of hesitation of the operation. The system calculates the included angle of the velocity vectors of adjacent sampling points, and if the included angle exceeds a preset threshold (e.g. 90 degrees), it is recorded as a sudden change of direction. The number of sudden changes in the window is counted and divided by the window length to obtain the sudden change frequency. High-frequency sudden changes in direction usually mean that the operator repeatedly advances and retreats in the target area, etc., which is a typical manifestation of psychological uncertainty.

[0042] Finally, the variance value and the sudden change frequency of the motion direction are weighted and fused to obtain the operation disturbance feature index representing the degree of hand tremor and the degree of operation hesitation of the operator. The fusion formula is: wherein is the acceleration variance, is the sudden change frequency of the direction, is a normalization function, and is a weight coefficient set according to expert experience (e.g. ). The index is a real-time changing scalar reflecting the roughness of the operator's operation at the current moment.

[0043] In specific calculation, in order to exclude the direction misjudgment caused by sensor noise in the state of hand stillness or very low speed, the system presets an effective speed threshold (preferably set to ). Only when the speed vector lengths of the instrument end at the moment and the moment are both greater than , the included angle is calculated by using the vector dot product formula. Wherein, represents the vector of the speed of the virtual surgical instrument end at the moment , and represents the vector of the speed of the virtual surgical instrument end at the moment (i.e. the last sampling moment) . If exceeds the preset mutation threshold (e.g. ), it is determined as an effective mutation.

[0044] In addition, in order to avoid the problem of denominator zero singularity caused by instrument static or slight jitter, the system sets direction mutation judgment logic, which is as follows: If or , the included angle is forcibly determined. Only when and , the above inverse cosine formula is used to calculate the included angle .

[0045] This logic ensures that the direction mutation is only counted when the main surgical action is performed, and the physiological hand tremor in the static state is excluded to the greatest extent.

[0046] In addition, the in the formula adopts a saturation normalization function based on expert benchmark data. It is defined as: . Wherein, is an interval limiting function, and its logic is: when the calculated value is less than , take , and when it is greater than , take , otherwise take the calculated value itself. Among them, for the acceleration variance, the limit threshold (based on the statistical upper limit of novice operation) is set; for the mutation frequency, the limit threshold is set. This processing ensures that the disturbance index of the input model is strictly constrained in the interval, preventing the model from being distorted due to a single feature value being too large.

[0047] Step S14: Establish a dynamic evolution mapping model of the operation disturbance feature index and the biomechanical parameters, dynamically correct the initial biomechanical parameters of the virtual thoracic tissue model according to the operation disturbance feature index calculated in real time, and generate an evolved virtual thoracic tissue model; when the operation disturbance feature index exceeds the preset safety interval, control the initial biomechanical parameters to deviate to a vulnerable state, so as to simulate the change of tissue pathological and physiological characteristics caused by rough operation in surgery.

[0048] Specifically, the step of establishing a dynamic evolution mapping model of the operation disturbance feature index and the biomechanical parameters, and dynamically correcting the initial biomechanical parameters includes: The first step is to define a cumulative damage factor (CDF). Considering that tissue damage usually has a cumulative effect, simply adjusting the parameter based on the instantaneous disturbance index may cause the system feedback to be too abrupt. Therefore, the system introduces a state variable CDF that is integrated over time. The cumulative damage factor increases nonlinearly with the increase of the operation disturbance characteristic index. The mathematical model is as follows: if the operation disturbance characteristic index at the current time is greater than the safety threshold , then , wherein is the cumulative rate coefficient, is a nonlinear index (usually greater than 1, indicating accelerated accumulation of damage). At the same time, in order to more realistically simulate the viscoelasticity of biological soft tissue, the model introduces an elastic recovery mechanism based on the time dimension. The system sets a recovery judgment window (e.g. 2000 ms), if the operation disturbance characteristic index of the operator in the window is continuously lower than the safety threshold , then the reverse decay logic of the damage factor is triggered: wherein is a self-healing coefficient (e.g. 0.01). This mechanism simulates the physiological process of gradual recovery of blood flow and reduction of edema in the compressed tissue when the surgeon stops rough action after realizing improper operation, thereby encouraging trainees to stop and adjust their mindset in time when they realize the risk. Wherein is always positive, ensuring that the power operation of the nonlinear index is valid in the real number domain. If , it is determined to be safe or recovered, and the reverse decay of is performed according to the aforementioned tissue viscoelastic recovery logic.

[0049] The second step is to use the cumulative damage factor to perform decay operation on the elastic modulus coefficient, so that the virtual thoracic tissue model exhibits tissue relaxation characteristics. In real surgery, repeated clamping will cause tissue edema, increase of intercellular fluid, and macroscopically, the tissue will become soft and lose its original resilience. The system dynamically updates the Young's modulus of the grid element according to the current CDF value through a decay function (such as exponential decay or S-shaped curve decay). As the value of decreases, the virtual tissue will produce greater deformation when subjected to the same external force, giving the operator an intuitive feeling of tissue damage.

[0050] Thirdly, the cumulative damage impact factor is used to perform a nonlinear attenuation operation on the tissue rupture threshold, so that the virtual thoracic tissue model is more likely to trigger the rupture logic when subjected to the same external force. With the increase of CDF, the ability of the tissue to withstand the ultimate stress decreases significantly. For example, the ultimate shear stress threshold of the blood vessel wall will decrease with the increase of CDF: wherein, represents the evolved ultimate shear stress threshold of the tissue at the current moment, represents the ultimate shear stress threshold set by the system initially, is defined as the rupture threshold attenuation constant, which is used to control the sensitivity of the increase of tissue brittleness with the accumulation of damage. In the embodiment, it is preferably set to . This value is set based on clinical experience, which means that when the cumulative damage impact factor reaches , the ability of the virtual blood vessel or alveolar wall to withstand the ultimate shear stress will decrease to (about ) of the initial state, simulating the real physical property that the diseased tissue is prone to rupture. After the damage accumulates to a certain extent, the pulling force that originally belongs to the safe range is now enough to cause the blood vessel to rupture and bleed.

[0051] In the preferred parameter setting of the embodiment, in order to truly simulate the biomechanical fatigue properties of the lung parenchyma tissue, the parameters of the dynamic evolution mapping model are set as follows: The safety threshold is set to , i.e. allowing the operator to have a physiological tremor of 20% amplitude; the cumulative rate coefficient in the cumulative damage calculation formula is set to , and the nonlinear index is set to . This means that once the operation is unstable beyond the threshold, the tissue damage will accumulate exponentially.

[0052] For dynamic correction of the elastic modulus coefficient, a hyperbolic tangent attenuation model is used to prevent parameter divergence: wherein, represents the dynamically corrected virtual tissue elastic modulus coefficient, represents the initial elastic modulus coefficient of the virtual tissue; the maximum softening coefficient is preferably . When the cumulative damage impact factor increases, the stiffness of the virtual tissue will approach 40% of the initial value, which macroscopically shows a significant decrease in compliance, i.e. the tissue collapses after being compressed and is difficult to rebound.

[0053] In addition, in order to enhance the multi-modal warning effect, the embodiment further includes a visual feedback step based on parameter evolution. The system monitors the change amplitude of the tissue rupture threshold in real time; when the tissue rupture threshold decreases below a preset dangerous value, the surface texture map of the virtual thoracic tissue model is automatically adjusted. This uses a multi-texture blending technique. The system presets the texture of normal tissue (pink, clear texture) and the texture of pathological tissue (dark red, congestion, edema gloss). In the pixel shader, the mixing weight is calculated according to the current CDF value, and the final texture color is synthesized in real time, so that the tissue surface presents a realistic congestion or edema visual effect. At the same time, a dynamic heat map is generated around the tip of the virtual surgical instrument, the color depth of the dynamic heat map is positively correlated with the current cumulative damage influence factor, and the semi-transparent red halo is projected on the operation area, directly prompting the operator that the area is in a high-risk state.

[0054] The visual effect is realized in the fragment shader of the graphics rendering pipeline. The system calculates the mixing weight according to the formula , where is defined as the visual pathology saturation threshold. In the embodiment, is set to . The physical meaning is that when the cumulative damage influence factor of the operator reaches or exceeds , the surface texture mixing weight of the virtual tissue reaches the saturation value , at which time the tissue visually presents a completely congested or severely edematous dark red state, and no longer changes color with the continued increase of , but its internal physical properties (such as the rupture threshold) can continue to deteriorate.

[0055] At the same time, in the tactile feedback control, when exceeds the safety value, the system superimposes a 150Hz sine wave micro-vibration in the force feedback signal. The frequency corresponds to the sensitive interval of human finger Pacinian corpuscles, which can make the operator clearly perceive the texture changes such as roughness of the tissue surface due to edema.

[0056] Step S15: According to the evolved virtual thoracic tissue model, the subsequent operation instruction data sequence of the operator is calculated for physical collision response and three-dimensional picture rendering by using a graphics rendering engine.

[0057] In this embodiment, when the physical parameters of the model are completed a round of dynamic correction, the system enters the physical calculation and rendering cycle. This process needs to be completed in a very short frame period (usually less than 16 ms) to ensure visual fluency.

[0058] Specifically, the step of performing physical collision response calculation and three-dimensional picture rendering on the subsequent operation instruction data sequence of the operator by using the graphics rendering engine comprises: A hierarchical bounding volume index structure (Bounding Volume Hierarchy, BVH) of the virtual thoracic tissue model is constructed. Due to the fact that the volume mesh model contains a large number of tetrahedral elements, the amount of calculation of point-surface collision detection is too large. The system uses an axis-aligned bounding box (AABB) tree or an oriented bounding box (OBB) tree to divide the space of the geometric body. When the virtual instrument moves, the physical engine first detects the intersection between the instrument bounding box and the root node of the tissue BVH tree. If there is no intersection, it is quickly eliminated. If there is an intersection, the child nodes are recursively traversed until the specific leaf node (i.e. the grid surface that may be in contact) is located.

[0059] Secondly, in each rendering frame period, the spatial pose vector set is used to traverse the hierarchical bounding volume index structure to detect whether the virtual surgical instrument and the virtual thoracic tissue model are in geometric contact. Once contact is detected, the narrow phase detection stage is entered, and the penetration depth, contact point normal vector and contact area are calculated.

[0060] If geometric contact is detected, the grid deformation variable of the contact area is calculated according to the current elastic modulus coefficient, and the image data in the display buffer is updated. At this time, the elastic modulus coefficient called by the physical engine is the evolved elastic modulus coefficient based on the finite element method or the position dynamics constraint solver. The displacement of the grid node under the action of the current external force is calculated. Since the elastic modulus has been reduced, the same force will cause a larger displacement, so that the tissue shows a clear relaxation feeling. At the same time, the system compares the current local stress with the evolved fracture threshold in real time. If the stress exceeds the threshold, the system triggers the topology change logic, i.e. disconnects the grid connection, generates a new free surface, and instantiates a particle system to simulate the effect of blood splashing, and finally completes the drawing and display of the picture by the graphics rendering engine.

[0061] The method further comprises a haptic feedback control step based on the evolved model: calculating reaction force data when the virtual surgical instrument cuts or pulls tissue according to the biomechanical parameters of the evolved virtual thoracic tissue model; converting the reaction force data into a force feedback control signal and sending it to the human-computer interaction interface. When the biomechanical parameters are in a vulnerable state, control the force feedback control signal to produce high-frequency micro-vibration to simulate the haptic texture difference of the pathological tissue. For example, when simulating edematous tissue, in addition to the reduction of feedback force, a weak sinusoidal vibration signal with a frequency of 50Hz-200Hz is superimposed to make the operator feel the roughness of the tissue surface, etc., thereby forming a haptic alarm.

[0062] Specifically, the system determines the current evolved elastic modulus coefficient Calculate the force feedback output The modified Hooke's law formula is used: Wherein is the shape factor of the contact surface, is the depth of the virtual instrument penetrating into the soft tissue, is the damping coefficient, is the relative penetration speed. Since the formula directly quotes the decreased with damage accumulation, The calculated feedback force will be reduced synchronously, so that the operator can clearly perceive the real haptic changes of the tissue caused by edema or relaxation and the loss of resistance at the handle end.

[0063] Step S16: Basic teaching evaluation step.

[0064] After the operation simulation is completed, the system needs to review the performance of the operator. The method of the embodiment further comprises: recording the total time consumed by the operator to complete the preset operation and the cumulative movement distance of the virtual surgical instrument; comparing the total time consumed and the cumulative movement distance with standard expert data to calculate a basic operation score; and generating a teaching analysis report containing the basic operation score. The standard expert data is the average trajectory and time data collected by multiple experienced thoracic surgery experts after the same operation in the system. By comparing the similarity of the student trajectory and the expert trajectory through the dynamic time warping (DTW) algorithm, and combining the historical curve of the operation disturbance index mentioned above, the system can accurately point out that the student has a mental fluctuation or operation error at the Xth minute of the operation, providing a quantitative basis for teaching.

[0065] Figure 2 The embodiment of the application further provides a thoracic surgery teaching auxiliary system, which integrates a high-performance computing unit, a professional-level display device and a force feedback interaction terminal in the hardware architecture. As shown in A virtual environment construction module is configured to construct a virtual surgery interaction environment containing a virtual thoracic tissue model endowed with initial biomechanical parameters. The module integrates a medical image processing engine and a physical modeling engine, responsible for DICOM data parsing, segmentation, 3D reconstruction, and tetrahedral mesh generation and attribute initialization.

[0066] A data acquisition module is configured to acquire a sequence of operator's operation instruction data for a virtual surgery instrument in real time through a human-computer interaction interface. The module contains a bottom-layer hardware driver and a signal preprocessing unit, responsible for managing high-frequency data transmission of a USB or FireWire interface, and performing a Kalman filtering algorithm to purify input signals.

[0067] A feature calculation module is configured to calculate an operation disturbance feature index of the operator in real time based on the sequence of operation instruction data. The module is a high-efficiency data stream processing unit, embedded with a sliding time window buffer queue and a statistical analysis algorithm, capable of outputting a current disturbance feature value with a microsecond-level delay.

[0068] A parameter evolution control module is configured to dynamically adjust biomechanical parameters of the virtual thoracic tissue model according to the operation disturbance feature index, to generate an evolved virtual thoracic tissue model. It maintains a tissue injury evolution model and a parameter mapping table, and controls dynamic changes of the virtual environment.

[0069] A rendering interaction module is configured to perform physical collision response calculation and 3D picture rendering using a graphics rendering engine according to the evolved virtual thoracic tissue model. The module integrates a physical simulation engine (such as PhysX or Bullet) and a graphics rendering pipeline, responsible for processing complex geometric collision, mechanical solution, light and shadow rendering, and haptic signal generation.

[0070] The system further includes a processor and a memory storing a computer program, the processor implementing the method steps of any one of claims 1 to 9 when executing the computer program. The processor can be a heterogeneous combination of a central processing unit (CPU) and a graphics processing unit (GPU), wherein the CPU is responsible for logical control and physical parameter evolution, and the GPU is responsible for large-scale parallel mesh deformation calculation and pixel rendering.

[0071] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A teaching aid method for thoracic surgery, characterized in that, The method includes the following steps: A virtual surgical interaction environment for thoracic surgery based on digital twin technology is constructed. The virtual surgical interaction environment includes a virtual thoracic tissue model that has been given initial biomechanical parameters, which at least define the elastic modulus coefficient and tissue rupture threshold of the virtual tissue. The operation instruction data sequence of the operator on the virtual surgical instrument is collected in real time through the human-computer interaction interface. The operation instruction data sequence includes the set of spatial pose vectors of the virtual surgical instrument on the time axis. Based on the operation instruction data sequence, the operator's operation disturbance characteristic index is calculated in real time using a preset sliding time window algorithm. The operation disturbance characteristic index is used to characterize the degree of dispersion of the operator's actions within a unit of time. A dynamic evolution mapping model between the operation disturbance characteristic index and biomechanical parameters is established. Based on the operation disturbance characteristic index calculated in real time, the initial biomechanical parameters of the virtual thoracic tissue model are dynamically corrected using the dynamic evolution mapping model to generate the evolved virtual thoracic tissue model. When the operation disturbance characteristic index exceeds a preset safe range, the initial biomechanical parameters are controlled to shift towards a vulnerable state to simulate the changes in tissue pathophysiological characteristics caused by rough operation during surgery. Based on the evolved virtual thoracic tissue model, a graphics rendering engine is used to perform physical collision response calculations and 3D rendering on the operator's subsequent operation command data sequence.

2. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The steps for constructing a virtual surgical interaction environment for thoracic surgery based on digital twin technology include: Acquire tomographic images of the patient's chest and use 3D reconstruction technology to generate a static mesh model containing lung lobes, bronchi, and pulmonary vessels; The vertices of the static mesh model are assigned mechanical property values, which are set based on physical measurement data of healthy human thoracic tissue. In the virtual surgical interactive environment, a virtual camera and virtual light source are configured to establish a rendering pipeline for simulating the thoracoscopic field of view.

3. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The step of collecting the operator's operation command data sequence for virtual surgical instruments in real time through a human-computer interaction interface includes: The six-degree-of-freedom sensor signal output from the human-machine interface is read at a preset high-frequency sampling rate; The six-degree-of-freedom sensor signal is processed by Kalman filtering to remove environmental noise interference, and the coordinate position and attitude angle of the end of the virtual surgical instrument are calculated to form the spatial pose vector set.

4. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The steps of using a graphics rendering engine to perform physical collision response calculations and 3D rendering of the operator's subsequent operation command data sequence include: Construct the hierarchical bounding box index structure of the virtual thoracic tissue model; During each rendering frame cycle, the spatial pose vector set is used to traverse the hierarchical bounding box index structure to detect whether the virtual surgical instrument and the virtual thoracic tissue model have geometric contact. If geometric contact is detected, the mesh deformation of the contact area is calculated based on the current elastic modulus coefficient, and the image data in the display buffer is updated.

5. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The method also includes basic instructional assessment steps: Record the total time taken for the operator to complete the preset surgical actions and the cumulative distance traveled by the virtual surgical instrument; The total time and the cumulative distance traveled are compared with standard expert data to calculate the basic operation score; Generate a teaching analysis report that includes the scores for the basic operations.

6. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The step of calculating the operator's operational disturbance characteristic index in real time based on the operational instruction data sequence using a preset time-series data analysis algorithm includes: Extract the kinematic feature sequence from the set of spatial pose vectors; Calculate the dispersion statistics of the kinematic feature sequence within the statistical window; Detect the trajectory smoothness index of the spatial pose vector set within a statistical window; The dispersion statistics index and the trajectory smoothness index are weighted and fused to obtain the operation disturbance characteristic index, which characterizes the degree of hand tremor and operational hesitation of the operator.

7. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The steps of establishing a dynamic evolution mapping model between the operational perturbation characteristic index and biomechanical parameters, and dynamically correcting the initial biomechanical parameters, include: Define a cumulative damage impact factor, which increases nonlinearly with the increase of the operational disturbance characteristic index; Configure tissue viscoelastic recovery logic. When the operation disturbance characteristic index is detected to be continuously within the safe range within a preset time period, control the cumulative damage influence factor to reverse back according to a preset time decay function in order to simulate the elastic self-repair characteristics of biological tissue. The cumulative damage influence factor is used to attenuate the elastic modulus coefficient, so that the virtual thoracic tissue model exhibits tissue relaxation characteristics. The cumulative damage influence factor is used to perform a nonlinear decay calculation on the tissue rupture threshold, making the virtual thoracic tissue model more likely to trigger the rupture logic when subjected to the same external force.

8. The teaching aid method for thoracic surgery according to claim 7, characterized in that, The method also includes a visual feedback step based on parameter evolution: Real-time monitoring of the changes in the tissue rupture threshold; When the tissue rupture threshold is reduced to below the preset danger value, the surface texture map of the virtual thoracic tissue model is automatically adjusted to overlay the visual effects of tissue congestion or edema. A dynamic heatmap is generated around the tip of the virtual surgical instrument, and the color depth of the dynamic heatmap is positively correlated with the current cumulative damage influence factor.

9. The teaching aid method for thoracic surgery according to claim 1, characterized in that, The method also includes a haptic feedback control step based on the evolved model: Based on the biomechanical parameters of the evolved virtual thoracic tissue model, calculate the reaction force data when the virtual surgical instrument cuts or pulls the tissue; The reaction force data is converted into a force feedback control signal and sent to the human-machine interface; When the biomechanical parameters are in a vulnerable state, the force feedback control signal is controlled to generate high-frequency micro-vibrations to simulate the tactile texture differences of diseased tissues.

10. A thoracic surgery teaching aid system, characterized in that, The system includes: The virtual environment construction module configures a virtual surgical interaction environment for building a virtual thoracic tissue model that has been given initial biomechanical parameters. The data acquisition module is configured to collect the sequence of operation instructions from the operator for the virtual surgical instruments in real time through the human-computer interaction interface; The feature calculation module is configured to calculate the operator's operation disturbance feature index in real time based on the operation instruction data sequence; The parameter evolution control module is configured to dynamically adjust the biomechanical parameters of the virtual thoracic tissue model according to the operational perturbation characteristic index, thereby generating the evolved virtual thoracic tissue model. The rendering interaction module is configured to perform physical collision response calculations and 3D rendering using a graphics rendering engine based on the evolved virtual thoracic tissue model. The system further includes a processor and a memory storing a computer program, wherein the processor executes the computer program to implement the steps of the method as described in any one of claims 1 to 9.

Images