A medical virtual simulation system and its simulation training method for basic medical skills

Abstract

This invention relates to a medical virtual simulation system and a method for simulating and training basic medical skills, belonging to the field of simulation medical technology. The method includes acquiring the original interaction signals from an input device; determining the category of the currently activated basic clinical operation; converting the original interaction signals into virtual spatial forces through a multimodal logic mapping matrix corresponding to the category; and using the virtual spatial forces to drive a virtual simulation device, causing the virtual simulation device to move on virtual biological tissues within a virtual space, thereby simulating and training the clinical basic operation of the specified category. This method eliminates the need to independently design dedicated underlying interaction logic and mechanical rules for different basic medical operations and corresponding virtual simulation devices, thus decoupling the basic medical operations from the virtual simulation devices and solving the problems of existing technologies.

Description

Technical Field

[0001] This invention relates to the field of simulation medical technology, and in particular to a medical virtual simulation system and a method for simulating and training basic medical skills. Background Technology

[0002] With the application of virtual reality (VR) and augmented reality (AR) technologies in medical education, VR / AR-based medical virtual simulation systems have become a key means of standardized practical training in clinical skills. This system relies on virtual human anatomical models, virtual simulation instruments (such as digital models of virtual scalpels, puncture needles, clamps, stethoscopes, etc.), force-sensing interaction devices, and a 3D rendering engine to highly reproduce various basic medical operations such as puncture, cutting, clamping, auscultation, and percussion. This effectively overcomes the limitations of traditional practical training, which relies on physical teaching aids, scarce case resources, and fixed training venues.

[0003] Current mainstream VR / AR medical virtual simulation systems generally adopt a single-skill, single-instrument customized development model. This means that separate underlying interaction logic and mechanical rules are designed for different basic medical operations and corresponding virtual simulation instruments. This development model results in independent underlying interaction logic for different virtual simulation instruments, leading to deep coupling between basic medical operations and virtual simulation instruments. This can easily lead to system architecture fragmentation, affecting system scalability and cross-platform portability, and severely restricting the large-scale application of VR / AR medical virtual simulation systems. Summary of the Invention

[0004] In view of this, it is necessary to provide a medical virtual simulation system and a method for simulating and training basic medical skills, in order to solve the problem that the existing medical virtual simulation system adopts a single-skill, single-instrument customized development model, which leads to deep coupling between basic medical operations and virtual simulation instruments.

[0005] To address the above problems, this invention provides a method for simulating and training basic medical skills based on a medical virtual simulation system, comprising: Acquire the raw interaction signals from the input device; Determine the category of the currently activated basic clinical procedure; The original interaction signal is converted into a virtual space force through a multimodal logic mapping matrix corresponding to the category. The virtual simulation device is driven by the force of the virtual space, so that the virtual simulation device moves on the virtual biological tissue in the virtual space to conduct simulated training of the basic clinical operations of the aforementioned category.

[0006] In one possible implementation, the method further includes: Based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological tissue, calculate the interaction influence factor between the virtual simulation device and the virtual biological tissue; The dynamic damping coefficient in the multimodal logic mapping matrix is ​​corrected using the interaction influence factor.

[0007] In one possible implementation, based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological tissue, an interaction factor between the virtual simulation device and the virtual biological tissue is calculated, specifically including calculating the interaction factor using the following formula: ; Wherein: Ω is a predetermined interaction region centered on the point of action of the virtual simulation device in the virtual biological tissue; p is a spatial position variable within the predetermined interaction region Ω; p tool η represents the coordinates of the point of action; K(·) is the interaction kernel, which is used to characterize the distribution weight of the mechanical action of the virtual simulation device in space; S(p) is the tissue stiffness field function of the virtual biological tissue; and η is the interaction influence factor.

[0008] In one possible implementation, the original interaction signal is converted into a virtual space force using a multimodal logical mapping matrix corresponding to the category. Specifically, this includes converting the original interaction signal into a virtual space force using the multimodal logical mapping matrix shown below: F virtual =M type ·Φ(I t )+B; Among them, F virtual M represents the virtual spatial force at time t; type This is the multimodal logic mapping matrix corresponding to the category; I t Let I be the original interactive signal received at time t; Φ(·) represents the original interactive signal I. t A nonlinear preprocessing function for denoising and smoothing; B is a biomechanical baseline bias preset based on the type of the virtual biological tissue.

[0009] In one possible implementation, the method further includes: based on the virtual space force F virtual The interaction results with the virtual biological organization update the interaction state, wherein the interaction state is represented by an interaction state tensor Ψ, which includes at least parameters G, F, and K, wherein G is based on F. virtual Updated instrument geometric pose; F is derived from F virtualThe tactile feedback torque generated by the interaction with the virtual biological tissue; K is the interaction kernel corresponding to the category.

[0010] In one possible implementation, the method further includes constructing a general interaction state operator and constructing a common feature space for uniformly representing multiple categories of basic clinical operations, wherein the interaction state tensor Ψ is defined in the common feature space.

[0011] In one possible implementation, the method further includes: using a multi-category unified evaluation functional to quantitatively evaluate the simulation training process of the clinical basic operation.

[0012] In one possible implementation, the multi-category unified evaluation functional specifically includes: ; Among them, Ψ t Let Ψ be the interaction state tensor at time t during the operation; std W is the standard interaction state tensor corresponding to the category; W is the dimension weight matrix corresponding to the category.

[0013] In one possible implementation, the category specifically includes puncture, cutting, clamping, auscultation, or percussion.

[0014] To address the aforementioned problems, the present invention also provides a medical virtual simulation system, which is used to conduct simulated training of basic medical skills using the methods provided in the embodiments of this application.

[0015] The beneficial effects of this invention are: Compared with existing technologies, the medical basic skills simulation training method based on a medical virtual simulation system provided in this embodiment includes acquiring the original interaction signals of the input device, determining the category of the currently activated basic clinical operation, then converting the original interaction signals into virtual spatial forces through a multimodal logic mapping matrix corresponding to the category, and then using the virtual spatial forces to drive the virtual simulation device, causing the virtual simulation device to move within a virtual space containing virtual biological tissues, thereby conducting simulation training for the category of basic clinical operations. This method, by introducing a multimodal logic mapping matrix corresponding to the category, enables the original interaction signals of the input device to be converted into virtual forces conforming to the physical characteristics of the operation, acting on the virtual simulation device, and thus conducting simulation training for the category of basic clinical operations. Therefore, this method does not require the separate design of exclusive underlying interaction logic and mechanical rules for different basic medical operations and corresponding virtual simulation instruments. It only needs to switch the corresponding multimodal logic mapping matrix to convert the original interaction signals for different categories of basic clinical operations, thereby decoupling the basic medical operations and virtual simulation instruments. This solves the problems of existing technologies, improves the scalability and cross-platform portability of medical virtual simulation systems, and facilitates the large-scale application of medical virtual simulation systems. Attached Figure Description

[0016] Figure 1 The present invention provides a flowchart of a method for simulating and training basic medical skills based on a medical virtual simulation system. Figure 2 This is a schematic diagram of the structure of the electronic device provided by the present invention; Figure 3 The present invention provides a schematic diagram of the structure of a medical basic skills simulation training device based on a medical virtual simulation system. Detailed Implementation

[0017] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0018] As mentioned earlier, current mainstream VR / AR medical virtual simulation systems generally adopt a single-skill, single-instrument customized development model. This means that separate underlying interaction logic and mechanical rules are designed for different basic medical operations and corresponding virtual simulation instruments. This development model results in independent underlying interaction logic for different virtual simulation instruments, leading to deep coupling between basic medical operations, virtual simulation instruments, and simulation function modules. This can easily lead to system architecture fragmentation, affecting system scalability and cross-platform portability, and severely restricting the large-scale application of VR / AR medical virtual simulation systems.

[0019] In view of this, embodiments of this application provide a medical virtual simulation system and its basic medical skills simulation training method, which can be used to solve this technical problem. For example... Figure 1 The diagram shown is a flowchart of a medical basic skills simulation training method based on a medical virtual simulation system, provided in an embodiment of this application. The method includes the following steps: Step S11: Acquire the raw interaction signal from the input device.

[0020] In the embodiments of this application, the input device may refer to a hardware device used to capture the user's hand movements or operating intentions, including but not limited to force feedback handles, force interaction devices, data gloves, position trackers, or dedicated surgical simulation controllers.

[0021] The raw interactive signal is the raw data stream output by the input device at the sampling moment. Its source can be voltage values ​​directly acquired by the sensor, digital pulses, or unprocessed coordinate data. The function of this raw interactive signal is to serve as the initial excitation source driving the virtual simulation environment. Its specific form varies depending on the type of input device; for example, it can be a displacement vector in three-dimensional space, a rotational quaternion, or an analog reading from a pressure sensor.

[0022] For example, when a user uses a force feedback handle to perform a simulated puncture in a medical virtual simulation system, the six degrees of freedom posture data and gripping force at the end of the force feedback handle can be collected at a frequency of 1000Hz, forming a raw data packet containing a timestamp as the raw interaction signal. The object collecting this raw interaction signal can be the input device or other devices within the medical virtual simulation system; no specific limitation is made here.

[0023] Step S12: Determine the category of the currently activated basic clinical operation.

[0024] Among them, the category of basic clinical operations can refer to the training item type preset by the system with specific physical interaction characteristics. For example, in practical applications, this category can specifically include one of puncture, cutting, clamping, auscultation or percussion.

[0025] This category can be determined either by explicit selection commands from the user on the graphical user interface (GUI) of the medical virtual simulation system, or by automatic identification based on the currently loaded virtual scene or the type of virtual device attached. The purpose of determining the operation category is to activate a specific set of physics engine parameters and logical judgment rules that match it, enabling the system to distinguish the differentiated mechanical response requirements of different operations.

[0026] For example, when a user selects the liver biopsy training module in the main menu of a medical virtual simulation system, or picks up a puncture needle in the virtual environment, the system identifies the currently active clinical basic operation category as puncture; if the user selects skin suturing or picks up a scalpel, it identifies it as cutting. By accurately determining the category of the currently active clinical basic operation, crucial contextual constraints can be provided for subsequent signal conversion processes, ensuring that different types of operations invoke the correct mapping strategy.

[0027] Step S13: Convert the original interaction signal into a virtual space force using the multimodal logic mapping matrix corresponding to the category.

[0028] The multimodal logical mapping matrix is ​​a pre-constructed mathematical transformation model that stores weighting coefficients, gain factors, and nonlinear correction parameters for different clinical basic operation categories. This model characterizes the mapping relationship from the input device signal domain to the virtual physical force domain. Typically, this multimodal logical mapping matrix can be obtained in advance based on statistical analysis and biomechanical modeling of a large amount of real clinical operation data.

[0029] It is important to note that this application constructs corresponding multimodal logic mapping matrices for different categories of basic clinical operations. In this way, in step S13, the multimodal logic mapping matrix corresponding to the category determined in step S12 can be obtained by switching or loading the mapping matrix, and then the original interaction signal can be converted into virtual space force through the multimodal logic mapping matrix.

[0030] For example, in the case of puncture, in step S13, the multimodal logic mapping matrix corresponding to the puncture category can be switched or loaded first, and then the original interaction signal can be converted into a virtual space force using the multimodal logic mapping matrix. Similarly, in the case of other categories, the corresponding multimodal logic mapping matrix can also be switched or loaded first, and then the original interaction signal can be converted into a virtual space force using the multimodal logic mapping matrix.

[0031] It should be further explained that step S13 can be implemented by converting the original interaction signal into a virtual space force through a multimodal logic mapping matrix as shown in Formula 1. F virtual =M type ·Φ(I t Formula 1 (+B) In Formula 1, F virtual M represents the virtual spatial force at time t; type This is the multimodal logic mapping matrix corresponding to the category determined in step S12; I tLet I be the original interactive signal received at time t; Φ(·) represents the original interactive signal I. t A nonlinear preprocessing function for denoising and smoothing; B is the biomechanical baseline bias preset based on the type of the virtual biological tissue.

[0032] Specifically, during the simulated training of basic medical skills using this medical virtual simulation system, the input device collects the original interactive signal I at time t (for example, t can be any value greater than or equal to 0, representing any time). t .

[0033] The original interactive signal I t Noise reduction and smoothing are achieved through a nonlinear preprocessing function Φ(·). The algorithm for the nonlinear preprocessing function Φ(·) can include exponential smoothing filtering, low-pass filtering, or Kalman filtering, etc., and the specific algorithm type is not limited here. For example, when using the exponential smoothing algorithm, Φ(I t )=α×I t +(1-α)×I t-1 , among which, I t-1 Let I be the original interactive signal at time t-1 (i.e., the time preceding time t), and α be a smoothing coefficient (ranging from 0 to 1) used to balance the real-time performance and stability of the signal. Then, the original interactive signal I is processed by the nonlinear preprocessing function Φ(·). t Nonlinear preprocessing ensures that the signal transmitted to the multimodal logic mapping matrix is ​​smooth and continuous, eliminating abnormal fluctuations in virtual instruments caused by hardware errors.

[0034] M type This is the multimodal logic mapping matrix corresponding to the category determined in step S12. Generally speaking, when the category is puncture, M... type It assigns a higher gain coefficient to the axial displacement component to simulate the sudden change in resistance when the needle penetrates the tissue; while in the case of auscultation, M type This focuses on positional accuracy mapping and reduces force gain to avoid false acoustic events. In Formula 1, the multimodal logic mapping matrix M... type When used in conjunction with the nonlinear preprocessing function Φ(·), the former defines the physical rules of the operation, while the latter ensures the purity of the input data. Together, they ensure a high-fidelity conversion from user actions to virtual feedback.

[0035] The biomechanical baseline bias B is a constant or variable vector based on a predefined type of virtual biological tissue, used to simulate the initial mechanical properties of different anatomical structures. For example, in real medical procedures, when instruments come into contact with different tissues (such as skin, fat, muscle, and bone), even without significant deformation, different basic resistance or tension will be felt. This biomechanical baseline bias B is designed to reproduce this realistic tactile experience of immediate contact.

[0036] Specifically, the biomechanical baseline bias B can be obtained from a lookup table based on the tissue type label corresponding to the current position of the instrument in virtual space. For example, when the virtual simulation instrument contacts the simulated dense fascia layer, B is set to a large positive resistance vector; while when it contacts the subcutaneous fat layer, B is set to a small resistance value or even zero. This mechanism enables the virtual space force F to... virtual It not only includes the dynamic forces actively applied by the user, but also superimposes the static mechanical properties of the environment itself, thereby significantly improving the anatomical realism of the simulation.

[0037] Therefore, in step S13 of this application, the original interactive signal I at time t can be... t Substituting these values ​​into Formula 1, the force F acting in the virtual space can be calculated. virtual .

[0038] Step S14: Use the force of the virtual space to drive the virtual simulation device, so that the virtual simulation device moves on the virtual biological tissue in the virtual space to conduct simulation training of the corresponding category of basic clinical operations.

[0039] The virtual simulation device can refer to a digital medical tool constructed in virtual space that has a geometric model and physical properties, such as a virtual puncture needle, a virtual scalpel, or a virtual stethoscope probe. The virtual biological tissue can refer to a digital human tissue model constructed in virtual space through finite element analysis or a point-spring model, possessing elasticity, viscosity, and deformation capabilities.

[0040] In step S14, the virtual space force drives the rigid body dynamics calculation of the virtual simulation device within the virtual space, causing it to undergo various movements such as displacement, rotation, and deformation that conform to physical laws. In practical applications, the execution entity of step S14 is the physics engine of the virtual simulation system, and its execution method is to apply the virtual space force F calculated in step S13. virtualThe force is applied to the center of the virtual simulation device, and collision detection and dynamic updates are performed in conjunction with the reaction force of the virtual biological tissue. The output is the real-time pose change of the virtual simulation device in virtual space and the accompanying tissue deformation effect, which in turn generates the visual rendering and the reverse driving torque required by the force feedback device.

[0041] For example, when the calculated virtual space force F virtual When the pressure exceeds the yield threshold of the virtual skin model, the virtual simulation instrument (puncture needle) will penetrate the virtual biological tissue (skin) and leave a puncture trajectory in the virtual space. At the same time, the force feedback device applies a corresponding change in resistance to the user's hand. Thus, users can complete a full simulation training of basic clinical procedures in the virtual space, obtaining a tactile experience and visual feedback close to that of real surgery, effectively solving the problem of poor training results caused by fragmented interactive logic in traditional systems.

[0042] The medical basic skills simulation training method based on a medical virtual simulation system provided in this application includes acquiring the original interaction signal of the input device, determining the category of the currently activated basic clinical operation, and then converting the original interaction signal into a virtual space force through a multimodal logic mapping matrix corresponding to the category. This virtual space force is then used to drive a virtual simulation device, causing the virtual simulation device to move within a virtual space containing virtual biological tissue, thereby simulating and training the clinical basic operation of that category. This method, by introducing a multimodal logic mapping matrix corresponding to the category, allows the original interaction signal of the input device to be converted into a virtual force conforming to the physical characteristics of the operation, acting on the virtual simulation device, and thus simulating and training the clinical basic operation of that category. Therefore, this method does not require independently designing dedicated underlying interaction logic and mechanical rules for different basic medical operations and corresponding virtual simulation devices. It only requires switching the corresponding multimodal logic mapping matrix to convert the original interaction signal for different categories of basic clinical operations, thereby decoupling the basic medical operations and the virtual simulation device, solving the problems of existing technologies, improving the scalability and cross-platform portability of the medical virtual simulation system, and facilitating the large-scale application of the medical virtual simulation system.

[0043] It should be further explained that the execution order of the method provided in this application embodiment can be executed in the order of steps S11 to S14 as described above, or step S12 can be executed first, followed by steps S11, S13 and S14, or steps S11 and S12 can be executed simultaneously, followed by steps S13 and S14, or other orders can be followed.

[0044] It should be further explained that, as mentioned in step S14 above, the virtual simulation device is driven by the virtual space force, causing it to move on virtual biological tissue within the virtual space, thereby conducting simulation training for corresponding categories of basic clinical operations. During the interaction between the virtual simulation device and the virtual biological tissue, to improve the simulation effect, the method may further include, based on the virtual space force F... virtual The interaction results with the virtual biological organization are used to update the interaction state, which is characterized by the interaction state tensor Ψ, which includes at least parameters G, F and K.

[0045] In this embodiment, the interaction state tensor Ψ is a structured mathematical model used to uniformly record the entire process data of the interaction between virtual simulation devices and virtual biological tissues during simulation training. The three parameters G, F, and K included in the interaction state tensor Ψ together constitute a complete snapshot of the physical environment at the current moment. Parameter G is based on F... virtual The updated mechanical geometry pose is derived from the virtual simulation engine's application of forces F in virtual space. virtual The output results after dynamic calculation are used to characterize the position coordinates and rotational attitude of the virtual simulation device in three-dimensional space; parameter F is derived from F virtual The tactile feedback torque generated by the interaction with the virtual biological tissue is obtained by calculating the coupling between the force applied by the virtual device to the virtual biological tissue and the reaction force generated by the tissue. This torque is used to drive the input device to present the user with a realistic sense of resistance or breakthrough. Parameter K is the interaction kernel corresponding to this category. It is a feature transformation operator called from a pre-set library based on the category of the currently activated basic clinical operation. It is used to define the spatial distribution weight or signal excitation mode of the mechanical action in the current scene. In this way, parameters G, F, and K work together. G provides the spatial reference, F provides the basis for mechanical feedback, and K provides physical rule constraints. Through the coordinated expression of the three, the physical consistency and integrity of the state description are ensured.

[0046] For example, during a puncture procedure, if F virtual This causes the needle tip to penetrate the skin layer, and the medical virtual simulation system immediately responds according to F... virtual G is updated to the new coordinates of the needle tip entering the subcutaneous tissue. At the same time, the step-like tactile feedback torque F is calculated based on the mechanical abrupt change at the moment of skin rupture, and K is locked as the axial interaction kernel representing the point contact resistance. This accurately records this critical moment of clinical action, thereby improving the simulation accuracy.

[0047] It should be further noted that, in order to improve the applicability of the method provided in the embodiments of this application to the simulation training of various categories of basic medical skills, the method may further include constructing a general interactive state operator and constructing a common feature space for uniformly representing multiple categories of basic clinical operations, wherein the aforementioned interactive state tensor Ψ can be defined in the common feature space.

[0048] Here, the general interactive state operator can refer to an abstract computational unit used to perform state evolution and data processing within a unified mathematical framework. Specifically, the general interactive state operator can be based on the virtual space force F obtained above. virtual The system generates tactile feedback torque F and interaction kernel K, and this general interaction state operator serves as the core driving engine to standardize and update heterogeneous medical operation data.

[0049] For example, this general interactive state operator can be configured to receive raw state data generated by different categories of basic clinical operations and map it to a preset vector space for computation. For instance, when the category of a basic clinical operation changes from auscultation to puncture, this general interactive state operator does not need to refactor the underlying code; it only needs to call a preset logic switch to convert the physical meaning of the acquired raw state data from acoustic feature excitation to axial drag calculation, thereby achieving instantaneous response to different skill modes. Therefore, by constructing this general interactive state operator, the medical virtual simulation system can process raw state data generated by multiple categories of basic clinical operations with a unified interface, effectively avoiding algorithm fragmentation caused by differences in the categories of basic clinical operations, and significantly improving the system's modularity and cross-platform reusability.

[0050] This common feature space can refer to an abstract multidimensional mathematical space whose dimensions are defined by the physical attributes shared by various categories of basic clinical operations. It is used to bridge the semantic gap between different operations. For example, this common feature space is set based on the common physical characteristics (such as spatial pose, mechanical feedback, and interaction mode) of five types of operations: puncture, cutting, grasping, auscultation, and percussion. In this common feature space, the instantaneous state of any category of basic clinical operation can be uniquely represented as an interaction state tensor Ψ. Specifically, whether the current operation is an auscultation operation requiring high-precision position tracking or a cutting operation requiring complex mechanical calculations, its state is projected into the same common feature space, forming a uniform tensor expression Ψ=[G, F, K]. For example, in this common feature space, the opposing closing force of the gripping action and the axial propulsive force of the piercing action, although physically different, are both mapped to different component distributions of the torque dimension F in this space. Therefore, by defining the interaction state tensor Ψ in this common feature space, the standardization of multi-skill state representation is realized, solving the technical problem in traditional technology that it is difficult to achieve unified modeling across skills due to different state representation methods.

[0051] It should be further explained that the multimodal logic mapping matrix in the embodiments of this application can be dynamically updated. Specifically, the interaction influence factor between the virtual simulation device and the virtual biological soft tissue can be calculated based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological soft tissue.

[0052] The interaction factor is a quantitative parameter characterizing the intensity of the interaction between the virtual simulation device and the virtual biological soft tissue. In practical applications, the calculation of this interaction factor relies on two core input variables: first, the real-time geometric pose of the virtual simulation device in virtual space, specifically including the three-dimensional coordinates, attitude angles, and motion velocity vectors of the device's end effector; these data are used to determine the specific point of contact and contact range between the device and the tissue; second, the physical properties of the virtual biological soft tissue, mainly referring to the stiffness field distribution function within the tissue, which quantifies the density or elastic modulus of the biological tissue at different locations within the virtual space.

[0053] Specifically, a medical virtual simulation system can define a predetermined interaction region centered on the point of action of a virtual simulated instrument in virtual biological soft tissue. Within this region, the system integrates the weighted distribution of the instrument's mechanical action with the tissue stiffness field to obtain an interaction factor. For example, when the tip of a puncture needle enters the calcified nodule region in a liver model, the tissue stiffness field function value in this region increases significantly, causing the calculated interaction factor to increase instantaneously, reflecting the presence of high resistance characteristics. Thus, through this dynamic calculation based on geometric pose and physical properties, the system can accurately capture the changes in the mechanical environment as the instrument moves between different tissue layers.

[0054] In practical applications, this interaction factor can be calculated using the following formula: Formula 2 In Formula 2, Ω represents a predetermined interaction area centered on the point of action of the virtual simulation instrument in the virtual biological tissue. This predetermined interaction area Ω is typically dynamically determined based on the physical dimensions of the virtual simulation instrument and a preset radius of action. For example, when the virtual simulation instrument is a puncture needle, the predetermined interaction area Ω can be set to the point centered on the needle tip coordinate p. tool The sphere is a spherical region with a center and a radius of 2mm. When the virtual simulation instrument is a scalpel, the predetermined interactive region Ω can be set as an elliptical cylindrical region extending along the blade direction.

[0055] p is a spatial location variable within the predetermined interaction region Ω. This spatial location variable p traverses all discrete grid points or continuous spatial points within the predetermined interaction region Ω and is used to collect local physical properties. tool为该 The coordinates of the point of action of the virtual simulation instrument in the virtual biological tissue, where the coordinates p of the point of action are... tool It can be obtained by tracking the six-degree-of-freedom pose of the input device in virtual space in real time, which serves as the reference center for integral calculation.

[0056] K(·) is the interaction kernel, which characterizes the spatial distribution weight of the mechanical action of the virtual simulation device. Its specific form depends on the category of the currently activated basic clinical procedure. For example, in a puncture procedure, K(pp tool K(pp) can be defined as a Gaussian distribution function, such that positions closer to the point of application have greater weights, thus simulating the concentrated force characteristics of point contact; in the gripping category, K(pp) tool It can be defined as a bimodal distribution function to characterize the symmetrical clamping force distribution on the tissue on both sides of the jaws.

[0057] S(p) is the tissue stiffness field function of the virtual biological tissue, which can be obtained based on a pre-set anatomical model of biological tissue in the virtual space. This tissue stiffness field function S(p) quantifies the stiffness characteristics of the tissue at different spatial locations. For example, when p is located in the skin surface, the value of S(p) is low; when p is located in the bone or calcified nodule area, the value of S(p) is significantly increased.

[0058] Therefore, the interaction factor η calculated by this formula is specifically obtained by spatially integrating the product of the aforementioned interaction kernel K(·) and the tissue stiffness field function S(p) within a predetermined interaction region Ω. This implicit function-based integral calculation method can accurately quantify the complex local mechanical interactions between the virtual simulation instrument and virtual biological soft tissue. For example, when the tip of the virtual simulation instrument moves from soft tissue to a hard nodule, the calculated interaction factor η will significantly increase due to the sharp increase in the value of S(p). This provides an accurate physical basis for subsequently correcting the dynamic damping coefficient in the multimodal logical mapping matrix, effectively simulating the breakthrough sensation during puncture or the resistance sensation when passing through different tissue layers.

[0059] After obtaining the interaction influence factor, it can be used to correct the dynamic damping coefficient in the multimodal logic mapping matrix. The dynamic damping coefficient is a key parameter in the multimodal logic mapping matrix used to adjust the signal conversion gain, directly determining the magnitude and response characteristics of the final generated virtual space force. Therefore, in this embodiment, the interaction influence factor can be used to correct the dynamic damping coefficient in the multimodal logic mapping matrix. The specific correction process involves using the calculated interaction influence factor as a feedback variable and dynamically inputting it into the update algorithm of the multimodal logic mapping matrix.

[0060] For example, when the interaction factor increases, the dynamic damping coefficient is automatically increased, so that the same original interaction signal is converted into a greater virtual resistance; conversely, when the interaction factor decreases, the dynamic damping coefficient decreases accordingly. In this way, by establishing a linkage mechanism between the interaction factor and the dynamic damping coefficient, the problem that fixed parameter mapping cannot adapt to complex tissue deformation is solved, so that the virtual simulation device can simulate the sense of resistance and breakthrough that conforms to the real physiological characteristics during the movement.

[0061] It should be further explained that the method provided in this application embodiment may further include using a multi-category unified evaluation functional to quantitatively evaluate the simulation training process of basic clinical operations. This multi-category unified evaluation functional can refer to a general mathematical evaluation model that is compatible with various types of basic clinical operations such as puncture, cutting, grasping, auscultation, and percussion. This multi-category unified evaluation functional unifies the evaluation framework, mapping operational data from different dimensions into comparable quantitative indicators.

[0062] In practical applications, this multi-category unified evaluation functional can be represented by the formula shown in Formula 3: Formula 3 In formula three, Ψ t Let Ψ be the interaction state tensor at time t during the operation; stdW is the standard interactive state tensor corresponding to the category determined in step S12; W is the dimension weight matrix corresponding to the category, used to adjust the weights of different state dimension errors in the multi-category unified evaluation functional J.

[0063] Therefore, when evaluating the operation at time t during the operation process, the interaction state tensor Ψ at time t during the operation process can be obtained. t The standard interaction state tensor Ψ corresponding to the category std .

[0064] For example, receiving the raw interactive signal I from the input device at time t. t The category of the currently activated basic clinical operation is determined, and the original interaction signal I is mapped using a multimodal logical mapping matrix corresponding to the category. t Converted into virtual space force F virtual Using virtual space force F virtual The virtual simulation device is driven to move within a virtual space containing virtual biological soft tissue. Furthermore, during operation, the interaction state tensor Ψ, representing the current operational state, is updated in real-time based on the interaction results between the forces in the virtual space and the virtual biological soft tissue. t (This content has already been explained above and will not be repeated here); Simultaneously, the standard interaction state tensor Ψ matching this category can be retrieved from the pre-set standard operation database. std This standard tensor records the ideal geometric pose, ideal haptic feedback torque, and ideal interaction kernel features of expert-level operations within the same time series; then, this Ψ t and the Ψ std Substituting these values ​​into Formula 3, we obtain the calculation result of the multi-category unified evaluation functional J.

[0065] Of course, the dimension weight matrix W in the multi-category unified evaluation functional J can be obtained based on the categories determined in step S12. In practical applications, the dimension weight matrix W can be a diagonal matrix or a sparse matrix, with its diagonal elements corresponding to the weight coefficients of different dimensions such as geometric pose G, haptic feedback torque F, and interaction kernel K in the interaction state tensor. The dimension weight matrix W can be used to evaluate the interaction state tensor Ψ at time t. t Its standard interaction state tensor Ψ std The difference vectors between the two are subjected to weighted norm operation to obtain the weighted error square term at time t; then, all weighted error square terms are accumulated and summed over the entire simulation training process from the initial time t=1 to the end time t=T to obtain the quantitative evaluation result, i.e., the value of the multi-class unified evaluation functional J.

[0066] Based on the method provided in the embodiments of this application, this application can also provide a medical virtual simulation system. This medical virtual simulation system is used for simulating training of basic medical skills using the medical virtual simulation system and its medical basic skills simulation training method provided in the embodiments of this application. Since this method can solve the problems in the prior art, this system can also solve the problems in the prior art, and will not be elaborated further here.

[0067] like Figure 2 As shown, the present invention also provides an electronic device 200, which includes a processor 201 and a memory 202. Figure 2 Only some of the components of the electronic device 200 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0068] In some embodiments, processor 201 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 202 or process data, such as the active insertion bypass method based on three-dimensional search in this invention. In some embodiments, processor 201 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 201 may be local or remote. In some embodiments, processor 201 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, etc., or any combination thereof.

[0069] In some embodiments, memory 202 can be an internal storage unit of electronic device 200, such as a hard disk or memory inside electronic device 200. In other embodiments, memory 202 can also be an external storage device of electronic device 200, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on electronic device 200. Furthermore, memory 202 may include both internal storage units and external storage devices of electronic device 200. Memory 202 is used to store application software and various types of data installed on electronic device 200.

[0070] The electronic device 200 also includes a computer program stored in a memory 202 and executable on a processor 201, which, when executed by the processor 202, implements the method described in this application.

[0071] Of course, the electronic device 200 may also include a display 203, which in some embodiments may be an LED display, a liquid crystal display, a touch-screen liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display 203 is used to display information from the electronic device 200 and to display a visual user interface. The components 201-203 of the electronic device 200 communicate with each other via a system bus.

[0072] This application also provides a computer-readable storage medium for storing a computer-readable program or instruction. When the program or instruction is executed by a processor, it can implement the steps or functions provided in the methods described above.

[0073] Based on the same inventive concept as the medical basic skills simulation training method based on a medical virtual simulation system provided in the embodiments of this application, the embodiments of this application also provide a medical basic skills simulation training device based on a medical virtual simulation system (hereinafter referred to as device 30). For any unclear aspects of device 30, please refer to the content in the embodiments of this application. Figure 3 The diagram shown illustrates the specific structure of the device 30, which includes an acquisition unit 301, a determination unit 302, a conversion unit 303, and a driving unit 304, wherein: Acquisition unit 301 is used to acquire the original interaction signal of the input device; Determining unit 302 is used to determine the category of the currently activated basic clinical operation; The conversion unit 303 is used to convert the original interaction signal into a virtual space force through a multimodal logic mapping matrix corresponding to the category; The driving unit 304 is used to drive the virtual simulation device using the force of the virtual space, so that the virtual simulation device moves on the virtual biological tissue in the virtual space to perform simulated training of the basic clinical operations of the aforementioned category.

[0074] Since the device 30 adopts the same inventive concept as the method provided in the embodiments of this application, and the device 30 can also solve the technical problem if the method can solve the prior art problem, this will not be elaborated here.

[0075] The device 30 may further include a correction unit for calculating the interaction influence factor between the virtual simulation device and the virtual biological tissue based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological tissue; and using the interaction influence factor to correct the dynamic damping coefficient in the multimodal logic mapping matrix.

[0076] Specifically, based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological tissue, the interaction influence factor between the virtual simulation device and the virtual biological tissue is calculated, including calculating the interaction influence factor using the following formula: ; Wherein: Ω is a predetermined interaction region centered on the point of action of the virtual simulation device in the virtual biological tissue; p is a spatial position variable within the predetermined interaction region Ω; p tool η represents the coordinates of the point of action; K(·) is the interaction kernel, which is used to characterize the distribution weight of the mechanical action of the virtual simulation device in space; S(p) is the tissue stiffness field function of the virtual biological tissue; and η is the interaction influence factor.

[0077] Specifically, the original interaction signal is converted into a virtual space force using a multimodal logical mapping matrix corresponding to the category. This includes converting the original interaction signal into a virtual space force using the multimodal logical mapping matrix shown below: F virtual =M type ·Φ(I t )+B; Among them, F virtual M represents the virtual spatial force at time t; type This is the multimodal logic mapping matrix corresponding to the category; I t Let I be the original interactive signal received at time t; Φ(·) represents the original interactive signal I. t A nonlinear preprocessing function for denoising and smoothing; B is a biomechanical baseline bias preset based on the type of the virtual biological tissue.

[0078] The device 30 may further include an update unit for updating based on the virtual spatial force F. virtual The interaction results with the virtual biological organization update the interaction state, wherein the interaction state is represented by an interaction state tensor Ψ, which includes at least parameters G, F, and K, wherein G is based on F. virtual Updated instrument geometric pose; F is derived from F virtual The tactile feedback torque generated by the interaction with the virtual biological tissue; K is the interaction kernel corresponding to the category.

[0079] The device 30 may also include a construction unit for constructing a general interaction state operator and a common feature space for uniformly representing multiple categories of basic clinical operations, wherein the interaction state tensor Ψ is defined in the common feature space.

[0080] The device 30 may also include an evaluation unit for quantitatively evaluating the simulation training process of the clinical basic operation using a multi-category unified evaluation functional.

[0081] Specifically, the multi-category unified evaluation functional includes: ; Among them, Ψ t Let Ψ be the interaction state tensor at time t during the operation; std W is the standard interaction state tensor corresponding to the category; W is the dimension weight matrix corresponding to the category.

[0082] The categories specifically include puncture, cutting, clamping, auscultation, or percussion.

[0083] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0084] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for simulating and training basic medical skills based on a medical virtual simulation system, characterized in that, include: Acquire the raw interaction signals from the input device; Determine the category of the currently activated basic clinical procedure; The original interaction signal is converted into a virtual space force through a multimodal logic mapping matrix corresponding to the category. The virtual simulation device is driven by the force of the virtual space, so that the virtual simulation device moves on the virtual biological tissue in the virtual space to conduct simulated training of the basic clinical operations of the aforementioned category.

2. The method according to claim 1, characterized in that, The method further includes: Based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological tissue, calculate the interaction influence factor between the virtual simulation device and the virtual biological tissue; The dynamic damping coefficient in the multimodal logic mapping matrix is ​​corrected using the interaction influence factor.

3. The method according to claim 2, characterized in that, Based on the geometric pose of the virtual simulation device and the physical properties of the virtual biological tissue, the interaction factor between the virtual simulation device and the virtual biological tissue is calculated, specifically by calculating the interaction factor using the following formula: ； Wherein: Ω is a predetermined interaction region centered on the point of action of the virtual simulation device in the virtual biological tissue; p is a spatial position variable within the predetermined interaction region Ω; p tool η represents the coordinates of the point of action; K(·) is the interaction kernel, which is used to characterize the distribution weight of the mechanical action of the virtual simulation device in space; S(p) is the tissue stiffness field function of the virtual biological tissue; and η is the interaction influence factor.

4. The method according to claim 1, characterized in that, The original interaction signal is converted into a virtual space force using a multimodal logical mapping matrix corresponding to the category. Specifically, this includes converting the original interaction signal into a virtual space force using the multimodal logical mapping matrix shown below: F virtual =M type ·Φ(I t )+B; Among them, F virtual M represents the virtual spatial force at time t; type This is the multimodal logic mapping matrix corresponding to the category; I t Let I be the original interactive signal received at time t; Φ(·) represents the original interactive signal I. t A nonlinear preprocessing function for denoising and smoothing; B is a biomechanical baseline bias preset based on the type of the virtual biological tissue.

5. The method according to claim 4, characterized in that, The method further includes: based on the virtual space force F virtual The interaction results with the virtual biological organization update the interaction state, wherein the interaction state is represented by an interaction state tensor Ψ, which includes at least parameters G, F, and K, wherein G is based on F. virtual Updated instrument geometric pose; F is derived from F virtual The tactile feedback torque generated by the interaction with the virtual biological tissue; K is the interaction kernel corresponding to the category.

6. The method according to claim 5, characterized in that, The method further includes constructing a general interaction state operator and constructing a common feature space for uniformly representing multiple categories of basic clinical operations, wherein the interaction state tensor Ψ is defined in the common feature space.

7. The method according to claim 1, characterized in that, The method further includes: using a multi-category unified evaluation functional to quantitatively evaluate the simulation training process of the clinical basic operation.

8. The method according to claim 7, characterized in that, The multi-category unified evaluation functional specifically includes: ； Among them, Ψ t Let Ψ be the interaction state tensor at time t during the operation; std W is the standard interaction state tensor corresponding to the category; W is the dimension weight matrix corresponding to the category.

9. The method according to claim 1, characterized in that, The categories specifically include puncture, cutting, clamping, auscultation, or percussion.

10. A medical virtual simulation system, characterized in that, The medical virtual simulation system is used to conduct simulated training of basic medical skills by means of the method described in any one of claims 1 to 9.

Images