AR-based animal heart catheter intubation teaching evaluation method and system

Abstract

The invention relates to the field of medical teaching software, and provides an AR-based animal heart catheter intubation teaching evaluation method and system, and the method comprises the steps: collecting user identity information, verifying the user identity information, and obtaining user authorized login data; according to the user authorized login data, identifying a scene where a user is currently located, and performing three-dimensional modeling on scene parameter information to obtain a virtual simulation teaching model; carrying out animal heart catheter intubation practical training operation through the virtual simulation teaching model, and meanwhile, carrying out recognition processing on a user operation gesture to obtain user feedback information; and matching a corresponding knowledge point assessment question group according to the user feedback information, and evaluating an answer result fed back based on the knowledge point assessment question group to obtain a learning evaluation result. According to the invention, AR-based teaching is realized, an evaluation mechanism is combined, support can be provided for personalized learning, and learning efficiency and teaching quality are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical teaching software, and particularly to an AR-based teaching evaluation method and system for animal cardiac catheterization intubation. Background Art

[0002] Augmented reality technology (AR) is a new computer application and human-computer interaction technology developed on the basis of virtual reality technology. AR technology superimposes entity information that is difficult to experience in the real world on the real environment through scientific and technological simulation processing by computers and the like. In recent years, AR technology has received more and more attention and development in the fields related to medicine.

[0003] Medical education has strong logic and has quite high requirements for students' practical ability and meticulous thinking. In the teaching of basic clinical skills, the following problems exist in the existing technology: Traditional experimental teaching requires the use of a large number of experimental animals, which not only increases the teaching cost, but also faces animal ethics issues, and cannot meet the needs of students for repeated practice; while two-dimensional plane teaching is difficult to accurately convey three-dimensional anatomical structures and complex operation skills, and students have difficulties in the spatial cognition of the heart structure and the understanding of surgical operations; in addition, although the early virtual simulation system provided three-dimensional models, the interaction method through keyboard and mouse operations is quite different from the actual surgical operation, which is difficult to cultivate students' practical operation skills, and the simulation degree is relatively low, and it is impossible to accurately simulate the physical characteristics during cardiac catheterization intubation, such as catheter elasticity, blood vessel resistance, and the physiological reaction of the heart after drug injection. At the same time, it is difficult for the existing technology to provide personalized teaching content and evaluation feedback according to the learning situation of students, and it is impossible to specifically improve the weak links of students, reducing the learning efficiency and flexibility. Summary of the Invention

[0004] The present invention provides an AR-based teaching evaluation method and system for animal cardiac catheterization intubation to solve the defects of the existing technology.

[0005] The present invention provides an AR-based teaching evaluation method for animal cardiac catheterization intubation, including the following steps: S1: Collect user identity information, verify the user identity information, and obtain user authorization login data; S2: Identify the current scene where the user is located according to the user authorization login data, perform three-dimensional modeling on the scene parameter information, and obtain a virtual simulation teaching model; S3: Perform animal cardiac catheterization intubation training operations through the virtual simulation teaching model, and at the same time identify and process the user's operation gestures to obtain user feedback information; S4: Match the corresponding knowledge point assessment question set according to the user feedback information, and evaluate the answer results based on the feedback of the knowledge point assessment question set to obtain the learning evaluation result.

[0006] According to an AR-based animal cardiac catheterization teaching and evaluation method provided by the present invention, step S2 further comprises: S21: Scan the environment through the camera of the AR device, and perform spatial positioning processing on the acquired environmental information to obtain three-dimensional spatial coordinate data; S22: calling a preset model library, fusing the preset model with the real environment according to the three-dimensional space coordinate data, and obtaining a basic simulation teaching model; the basic simulation teaching model includes a three-dimensional heart model and an animal cardiac catheterization surgery experimental model; S23: The transparency of the virtual simulation teaching model is adjusted by a deep learning algorithm to obtain a virtual simulation teaching model with a switching function between a physical display mode and a transparent display mode.

[0007] According to an AR-based animal cardiac catheterization teaching and evaluation method provided by the present invention, the step of acquiring the three-dimensional heart model in step S22 includes: S2211: collecting real heart anatomical data through a three-dimensional modeling tool, reconstructing the anatomical data, and obtaining an initial three-dimensional model of the heart; S2212: Establish the spatial topological structure of the heart cavity based on the initial three-dimensional model of the heart, plan the path of the heart and blood vessels, and obtain the dynamic simulation data of blood flow; S2213: Perform heart beating simulation on the blood flow dynamic simulation data through a physical engine to obtain a three-dimensional heart model with physiological cycle changes.

[0008] According to an AR-based animal cardiac catheterization teaching and evaluation method provided by the present invention, the step of obtaining the animal cardiac catheterization surgery experimental model in step S22 includes: S2221: collecting real operating table data and surgical instrument data through a three-dimensional scanning device, and performing mesh optimization on the operating table data and the surgical instrument data to obtain a three-dimensional model of the operating table and a three-dimensional model of the surgical instrument respectively; S2222: Construct the animal body structure based on the animal anatomical data, perform hierarchical processing on the internal organs of the animal, and obtain the animal three-dimensional model; S2223: Model the animal arterial-cardiac vascular system through computational fluid dynamics algorithms to obtain an animal cardiac catheterization surgery experimental model.

[0009] According to an AR-based animal cardiac catheterization teaching and evaluation method provided by the present invention, step S3 further comprises: S31: Perform animal cardiac catheterization training operations through the virtual simulation teaching model to obtain first feedback information; S32: Detect the user's finger movements through a gesture recognition algorithm, analyze the action types, and obtain gesture operation types; where the gesture operation types include direct click, single - hand pinching, single - hand rotation, double - hand rotation, and double - hand scaling; S33: Trigger the corresponding model transformation function according to the gesture operation type, perform transformation matrix calculation processing on the virtual simulation teaching model, and obtain the transformation effect of the model; where the transformation effect includes translation effect, rotation effect, and scaling effect; S34: Perform determination processing on the interaction between the user's gesture and the UI interface through physical collision detection, obtain second feedback information on the change in the selection state of UI elements, and output the first feedback information and the second feedback information as user feedback information.

[0010] According to an AR - based animal cardiac catheterization teaching evaluation method provided by the present invention, step S31 specifically includes: Simulate the physical characteristics of a real catheter through a virtual catheter control algorithm, perform real - time calculation on the catheter deformation, and obtain elastic deformation feedback; Perform collision detection processing on the catheter advancement path according to the vascular anatomical structure in the virtual simulation teaching model to obtain a tactile feedback signal; Perform physiological response calculation processing on the injection of different drugs through a drug reaction simulation system to obtain data on changes in cardiac function parameters.

[0011] The present invention also provides an AR - based animal cardiac catheterization teaching evaluation system, including: A head - mounted glasses - type virtual reality all - in - one machine, used to identify the user's current scene and collect environmental parameter information; A three - dimensional model processing module, used to generate a virtual simulation teaching model according to the environmental parameter information; A user interaction module, used to receive the user's operation gestures and generate an interaction instruction set; A user feedback module, used to evaluate the interaction operation results based on the interaction instruction set and generate a learning evaluation result.

[0012] According to an AR - based animal cardiac catheterization teaching evaluation system provided by the present invention, the head - mounted glasses - type virtual reality all - in - one machine includes: A binocular stereo vision camera, used to collect real - environment images and calculate depth information; An attitude sensor group, used to monitor head movements and update the perspective information in real time; A supporting handle device, used to provide spatial positioning and support key input; A perspective display screen for fusing and displaying a real environment and virtual elements.

[0013] According to an AR-based animal cardiac catheterization teaching and evaluation system provided by the present invention, the three-dimensional model processing module includes: A model library storage unit for storing a three-dimensional model of the heart and an experimental model of animal cardiac catheterization surgery; A rendering engine for generating various display modes of solid images and transparent images; A physical collision engine for simulating the physical characteristics of the interaction between a catheter and blood vessels; A drug reaction simulator for calculating the influence effects of different drugs on cardiac function parameters.

[0014] According to an AR-based animal cardiac catheterization teaching and evaluation system provided by the present invention, the user feedback module includes: An examination question bank database for storing a set of multiple-choice questions corresponding to knowledge points; A scoring and analysis unit for generating evaluation data according to operation accuracy and answering situations; A learning progress tracker for recording the learning trajectory and knowledge mastery degree of a user; A feedback report generator for outputting a learning report including short-term goals and long-term suggestions.

[0015] The present invention provides an AR-based animal cardiac catheterization teaching and evaluation method and system. The high simulation of the virtual simulation model greatly improves the students' spatial cognitive ability of the three-dimensional structure of the heart, reduces the dependence on experimental animals, effectively solves the problem of tight teaching resources, and at the same time meets the ethical requirements of animal protection. The immersive interactive experience provided by the head-mounted AR device enables students to achieve real-time interaction with the virtual model through intuitive gesture operations, significantly improving the training efficiency of operation skills. Especially in the learning of cardiac catheterization technology, the system accurately simulates the elastic deformation of the catheter and blood vessel resistance, and students can obtain tactile feedback similar to real surgery, greatly reducing the failure rate of actual operations. The drug reaction simulation function allows students to clearly observe the immediate effects of different drugs on cardiac function, closely combines abstract pharmacological knowledge with intuitive physiological reactions, greatly enhancing the memory effect and knowledge application ability. The personalized learning evaluation system can accurately track the learning process of students, automatically identify knowledge blind spots, provide targeted learning suggestions, greatly improve the learning efficiency, and shorten the skill mastery cycle.

[0016] The present invention breaks through the time and space limitations of traditional medical teaching. Students can conduct high-quality autonomous learning and skills training anytime and anywhere, significantly improving the utilization efficiency of teaching resources. The entity and transparency display switching function realized by three-dimensional modeling and deep learning algorithms enables students to understand the relevance between the heart structure and function from multiple angles and levels, forming a systematic knowledge network. It can instantaneously evaluate user feedback, and the guiding function makes the learning process more efficient and targeted, significantly enhancing the depth and breadth of knowledge mastery. The combination of the physical collision engine and the drug reaction simulator in the practical operation session enables students to experience a nearly real surgical operation feeling in a virtual environment, greatly reducing the trial-and-error costs and risks in actual operations.

[0017] The present invention significantly improves the quality and efficiency of medical education, cultivates students' innovative thinking and practical abilities, provides a new and effective path for the cultivation of medical talents, and at the same time lays a solid foundation for the application of AR technology in a wider medical education field. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic flowchart of a method for teaching and evaluating animal cardiac catheterization based on AR provided by an embodiment of the present invention; Figure 2 It is a schematic structural diagram of a system for teaching and evaluating animal cardiac catheterization based on AR provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. They should not be construed as limitations on the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention. In the description of the present invention, it should be understood that the terms used are only for the purpose of description and cannot be construed as indicating or implying relative importance.

[0021] As Figure 1 shown, the present invention provides a method for teaching and evaluating animal cardiac catheterization based on AR, including: S1: Collect the user identity information, verify the user identity information, and obtain the user authorized login data.

[0022] The present invention first obtains and stores the user identity information and verifies it to ensure that only authorized users can access the system. In practical applications, when students need to use the system, they first need to register and verify their identities. The system will pre-collect basic information such as student IDs and names, store them in the database, and generate a unique login credential. When students need to use the system for teaching, they only need to enter their account passwords, and the system will compare the information in the database. Only after successful verification can they enter the teaching environment.

[0023] S2: Identify the current scene where the user is located according to the user authorized login data, perform three-dimensional modeling on the scene parameter information, and obtain the virtual simulation teaching model.

[0024] Step S2 is carried out after obtaining the user authorized login data, aiming to fuse the actual scene with the virtual model through complex three-dimensional modeling technology, so as to create a highly realistic teaching environment.

[0025] Among them, step S2 further includes: S21: Scan the environment through the AR device camera, and perform spatial positioning processing on the obtained environment information to obtain three-dimensional space coordinate data.

[0026] Furthermore, the camera of the AR device (a head-mounted virtual reality all-in-one machine with a supporting handle) first scans the environment where the user is located in all directions, collects the visual information of the environment. When collecting, the camera captures the image data of the surrounding environment at a speed of at least 30 frames per second. Subsequently, through the spatial positioning processing technology, these two-dimensional image data are converted into three-dimensional space coordinate data.

[0027] Spatial positioning processing includes three key links: feature point extraction, feature point matching, and spatial reconstruction. Specifically, feature point extraction is to use the SLAM (Simultaneous Localization and Mapping) algorithm to identify obvious corner points, edges and other features in the image; feature point matching is to determine the corresponding relationship between them by comparing the similarity of feature points between consecutive frames; spatial reconstruction is based on the principle of triangulation to calculate the actual position coordinates of these feature points in three-dimensional space. Finally, the AR device can obtain an accurate three-dimensional space coordinate system, providing a basis for the placement of subsequent virtual models.

[0028] S22: Call the preset model library, fuse the preset model with the actual environment scene according to the three-dimensional space coordinate data, and obtain the basic simulation teaching model; the basic simulation teaching model includes a three-dimensional heart model and an animal cardiac catheterization surgery experiment model.

[0029] In the embodiments of the present invention, rabbits are specifically used as the above-mentioned experimental animals for illustration.

[0030] Further, in step S22, professional medical models such as a three-dimensional heart model and a rabbit cardiac catheterization surgery experimental model are stored in the preset model library, and the models all have a high degree of anatomical accuracy. During the fusion process, the system first determines the placement position of the virtual model according to the three-dimensional space coordinate data, and then performs transformations such as translation, scaling, and rotation on the model to make its proportion and direction consistent with the actual environment. Finally, through light simulation and shadow processing, the virtual model looks more natural and real, thereby obtaining the basic simulation teaching model.

[0031] Among them, the steps for obtaining the three-dimensional heart model in step S22 include: S2211: Collect real heart anatomical data through three-dimensional modeling tools, and perform image reconstruction on the anatomical data to obtain an initial three-dimensional heart model.

[0032] First, in S2211, the present invention pre-uses a medical CT or MRI scanning device to obtain tomographic images of the heart to capture the fine structure of the heart. The obtained tomographic image data can form a complete heart structure data set. Subsequently, image reconstruction is performed on these anatomical data, including image enhancement, edge detection, and region segmentation. Image enhancement is to make the heart structure clearer in the image by adjusting the contrast and brightness. Edge detection is to use operators such as Sobel and Canny to identify the boundary lines in the image; region segmentation is to separate each part of the heart (such as the atrium, ventricle, blood vessel, etc.) from the background. Through these processes, an initial three-dimensional heart model is finally generated, and the model contains the geometric information of the external shape and internal structure of the heart.

[0033] S2212: Establish a cavity space topological structure based on the initial three-dimensional heart model, perform path planning on the blood vessel directions of the heart, and obtain blood flow dynamic simulation data.

[0034] Further, in S2212, a cavity space topological structure is established based on the initial three-dimensional heart model. The topological structure describes the connection relationships between each part of the heart. For example, the left atrium is connected to the left ventricle through the mitral valve, and the right atrium is connected to the right ventricle through the tricuspid valve, etc. When establishing the topological structure, first perform skeleton extraction on the heart model, identify the centerlines of the heart cavities and major blood vessels, and establish the connection relationships between them. At the same time, perform path planning on the blood vessel directions of the heart. In this embodiment, the Dijkstra path search algorithm is used to determine the optimal path for blood flow in the heart. On this basis, by calculating the computational fluid dynamics equation, the flow state of blood in the heart is simulated, and blood flow dynamic simulation data is obtained. The data includes parameters such as blood flow velocity and pressure distribution, which can be used for subsequent heart beating simulation.

[0035] S2213: Simulate the heart beating of the blood flow dynamic simulation data through a physical engine to obtain a three-dimensional heart model with physiological cycle changes.

[0036] Furthermore, the physical engine is a software system capable of simulating various phenomena in the physical world. In the present invention, it is mainly used to simulate the elastic deformation of heart tissues and the contraction and relaxation processes of heart muscles. The heart beating simulation is based on the physiological cycle of the heart. A complete cardiac cycle is divided into two main stages: the systolic phase and the diastolic phase, and the changes in the heart shape in each stage are accurately simulated.

[0037] In the present invention, by dividing the heart model into multiple grid cells and assigning corresponding physical parameters (such as elastic coefficient, damping coefficient, etc.) to each cell, the physical engine can calculate the deformation of each part of the heart under the action of blood flow pressure. Finally, a three-dimensional heart model with physiological cycle changes is generated, which can truly display the morphological changes of the heart in different physiological states.

[0038] Among them, the steps for obtaining the experimental model of the animal cardiac catheterization surgery in step S22 include: S2221: Collect real operating table data and surgical instrument data through a three-dimensional scanning device, and perform grid optimization on the operating table data and surgical instrument data to obtain a three-dimensional operating table model and a three-dimensional surgical instrument model respectively.

[0039] In S2221, collect real operating table and surgical instrument data through a three-dimensional scanning device. The three-dimensional scanning device uses structured light scanning technology, which can quickly obtain the three-dimensional shape information of an object. For the operating table, the scanning accuracy is at the 0.1 mm level to ensure that the obtained data is accurate enough; for surgical instruments (such as catheters, syringes, etc.), since their shapes are more complex and have more details, the scanning accuracy needs to reach the 0.05 mm level.

[0040] The original data obtained by scanning is point cloud data. Subsequently, the present invention performs grid optimization processing on this point cloud data, including steps such as point cloud filtering, surface reconstruction, and mesh simplification. Point cloud filtering is to remove noise points and outliers; surface reconstruction is to convert point cloud data into a continuous triangular mesh, and mesh simplification is to reduce the number of triangles while keeping the model shape unchanged to improve the rendering efficiency. Through processing, an accurate three-dimensional operating table model and a three-dimensional surgical instrument model can be obtained.

[0041] S2222: Construct the animal body structure according to animal anatomy data, hierarchically process the internal organs of the animal to obtain a three-dimensional animal model.

[0042] Furthermore, the animal anatomy data mainly comes from actual rabbit anatomy materials, including information such as the skeletal structure, muscle distribution, and internal organ positions of rabbits. Based on these data, a rabbit skeletal system is first established as the framework of the entire model, and then the muscle system, circulatory system, digestive system, respiratory system, etc. are added in sequence to form a complete rabbit anatomy model. During this process, hierarchical processing is performed on the internal organs of the rabbit to establish a clear organizational structure relationship from the surface to the deep layer, enabling convenient display and hiding of organizational structures at different layers in subsequent AR teaching. The finally obtained three-dimensional rabbit model not only has an accurate appearance, but also its internal structure highly conforms to the actual anatomical situation, providing a real virtual environment for cardiac catheterization operations.

[0043] S2223: Model the animal arterial-heart vascular system through computational fluid dynamics algorithms to obtain an animal cardiac catheterization surgery experimental model.

[0044] In step S2223, CFD is mainly used to simulate the blood flow in the rabbit heart and blood vessels, including parameters such as flow velocity distribution and pressure change. The modeling process first requires constructing accurate geometric models of the rabbit heart and arterial blood vessels mainly including the aorta and pulmonary artery, especially the pulmonary artery, and setting the boundary conditions of the blood vessel wall. Then, the geometric space is discretized into a large number of grid cells. Finally, the hydrodynamic equations in each grid cell are solved to obtain the blood flow state in the entire blood vessel system. The CFD-based blood vessel model of the present invention can truly reflect the impact of the catheter on blood flow during insertion and the reaction force of the blood vessel wall on the catheter, thus providing a highly simulated operation experience.

[0045] S23: Adjust the transparency of the virtual simulation teaching model through deep learning algorithms to obtain a virtual simulation teaching model with the switching function between the entity display mode and the transparent display mode.

[0046] In step S23, the transparency of the virtual simulation teaching model is adjusted through deep learning algorithms. The deep learning algorithms mainly refer to image processing technologies based on convolutional neural networks (CNNs) here, which can automatically identify different organizational structures in the model and assign appropriate transparency values to them.

[0047] During the training process, CNN learns the characteristics of tissues through a large number of annotated medical images, forming the ability to distinguish different tissue types. When applied, CNN first classifies each part of the model to identify whether it is bone, muscle, blood vessel or other tissue, and then sets different transparency values for each tissue according to the preset transparency rules. Finally, the switching function between the solid display mode and the transparent display mode is realized. Users can switch freely between the two modes as needed to meet different teaching needs. The solid display mode is mainly used to display the external morphology of organs, and the transparent display mode is mainly used to observe the internal structure and blood vessel direction.

[0048] S3: Perform animal cardiac catheterization training operation through the virtual simulation teaching model, and recognize and process the user's operation gestures to obtain user feedback information.

[0049] Wherein, step S3 further comprises: S31: Performing a rabbit cardiac catheterization practice operation through the virtual simulation teaching model to obtain first feedback information.

[0050] Wherein, step S31 specifically includes: The virtual catheter control algorithm simulates the physical characteristics of the real catheter, calculates the catheter deformation in real time, and obtains elastic deformation feedback.

[0051] Furthermore, the virtual catheter control algorithm accurately simulates the physical properties of the real catheter and calculates the bending, twisting and other deformations of the catheter during insertion in real time. Based on the principles of elastic mechanics, the algorithm discretizes the catheter into multiple nodes, connects each node through a spring model, calculates the interaction force between nodes, and obtains elastic deformation feedback of the catheter. For example, when the user pushes the catheter with a virtual hand, the system calculates the friction, propulsion and resistance of the catheter, and calculates the degree and direction of the catheter's bending in the blood vessel based on the combined force of these forces, thereby generating realistic visual feedback.

[0052] According to the vascular anatomical structure in the virtual simulation teaching model, collision detection is performed on the catheter advancement path to obtain a tactile feedback signal.

[0053] Furthermore, according to the preset vascular anatomical structure in the virtual simulation teaching model, the system will perform collision detection processing on the catheter advancement path. The collision detection algorithm is based on the geometric models of the inner wall of the blood vessel and the catheter head, and determines whether a collision occurs by calculating the minimum distance between the two. When a collision event is detected, the system will generate a haptic feedback signal to simulate the resistance feeling of the catheter touching the blood vessel wall. The haptic feedback signal causes the virtual hand to vibrate slightly, enabling the user to perceive the physical contact in the virtual environment. For example, when the catheter tip touches the blood vessel bifurcation, the system detects the collision, immediately calculates the magnitude and direction of the reaction force, and feeds this force back to the controller in the user's hand, enabling the user to feel an obvious resistance, indicating that the catheter has touched the blood vessel wall.

[0054] The drug response simulation system performs physiological response calculation processing on different drug injections to obtain data on changes in cardiac function parameters.

[0055] The drug response simulation system is responsible for calculating and processing the physiological responses of rabbits after different drug injections. The system has built-in pharmacological models of various commonly used drugs and can calculate the changes in parameters representing cardiac function, such as heart rate, blood pressure, cardiac output, left ventricular pressure, and the rate of change of left ventricular pressure, according to the type, dose, and administration route of the drug. The parameter changes are displayed in real time on the virtual monitor, and the drug effects are visually demonstrated through the pulsation frequency and intensity changes of the three-dimensional heart model. The drug response simulation system uses a mathematical model that combines pharmacokinetics and pharmacodynamics to quantify the absorption, distribution, metabolism, and excretion processes of drugs in the body, thereby accurately calculating the impact of drugs on cardiac function.

[0056] S32: Detect the user's finger movements through a gesture recognition algorithm, analyze the movement types, and obtain the gesture operation types; among which the gesture operation types include direct click, single-hand pinching, single-hand rotation, two-hand rotation, and two-hand scaling.

[0057] The gesture recognition algorithm is responsible for detecting and analyzing the user's finger movements to obtain the type of gesture operation. The gesture recognition algorithm captures the user's hand movements through the depth camera of the virtual reality all-in-one machine, and then inputs this movement data into a pre-trained neural network model for classification and recognition. The system can recognize multiple basic gesture operation types: direct click, single-handed pinch, single-handed rotation, two-handed rotation, and two-handed zoom. The direct click gesture is determined by detecting the forward extension and stay time of the user's index finger. When the index finger stays at a specific position for more than the preset threshold, the system determines it as a click operation; the single-handed pinch gesture is recognized by detecting the distance change between the index finger and the thumb. When the distance between the two fingers changes from large to small and becomes less than a certain threshold, the system recognizes the start of the pinch action; single-handed rotation or two-handed rotation is recognized by identifying the hand position and the rotation angle; the two-handed zoom gesture requires tracking the positions of the index fingers and thumbs of both hands simultaneously and calculating the distance change between the two hands to determine the intention of zooming in or out.

[0058] S33: Trigger the corresponding model transformation function according to the gesture operation type, perform transformation matrix calculation processing on the virtual simulation teaching model, and obtain the transformation effect of the model; where the transformation effect includes translation effect, rotation effect, and scaling effect.

[0059] Furthermore, the transformation matrix calculation uses the homogeneous coordinate system in 3D graphics. The model is transformed by constructing a translation matrix, a rotation matrix, and a scaling matrix. When the system detects a direct click gesture, it will perform a selection operation to activate the clicked model or UI element. When a single-handed pinch gesture is detected, the system will calculate the translation vector according to the movement trajectory of the hand, construct a translation matrix, and adjust the position of the model to achieve the translation effect; when the user performs a rotation operation, the system will calculate the rotation axis and rotation angle, construct a rotation matrix, and make the model rotate around a specific axis by a specific angle to achieve the rotation effect; when a two-handed zoom gesture is detected, the system will calculate the scaling factor according to the change ratio of the distance between the two hands, construct a scaling matrix, and adjust the size of the model to achieve the scaling effect.

[0060] S34: Determine and process the interaction between the user's gesture and the UI interface through physical collision detection, obtain the second feedback information on the change of the selection state of the UI element, and output the first feedback information and the second feedback information as user feedback information.

[0061] The present invention also determines and processes the interaction between the user gesture and the UI interface through physical collision detection. Ray casting technology is used for UI interface collision detection. A virtual ray is emitted from the user's perspective or finger position, the intersection point of the ray and the UI plane is calculated, and it is determined whether the intersection point falls within the boundary of a certain UI element. When it is detected that the user's virtual finger or controller pointer collides with a UI element, the system changes the visual state of the UI element (such as highlighting), and when the user performs a confirmation operation (such as clicking), the corresponding UI event is triggered. This interaction method simulates the touch operation in the real world, enabling the user to intuitively interact with the virtual interface.

[0062] Finally, in step S3 of the present invention, the first feedback information (including the feedback of catheter elastic deformation, tactile feedback signal, and the change data of cardiac function parameters) and the second feedback information (the change of UI element selection state) are integrated and output as complete user feedback information. These feedback information are synchronously transmitted to the user through multiple sensory channels of vision, audition, and touch, providing the user with an immersive learning experience.

[0063] S4: Match the corresponding knowledge point assessment question groups according to the user feedback information, and evaluate the answering results based on the knowledge point assessment question groups to obtain the learning assessment results.

[0064] In step S4, the corresponding knowledge point assessment question groups are matched according to the user feedback information, and the answering results based on the knowledge point assessment question groups are evaluated. Finally, the learning assessment results are obtained. Through multiple technical links such as matching algorithms, scoring mechanisms, and data integration in step S4, an objective evaluation of the mastery degree of knowledge points is carried out.

[0065] The matching of knowledge point assessment question groups is an accurate data processing process. First, it is necessary to analyze and classify the user feedback information. The user feedback information includes two parts: the first feedback information (the feedback of catheter elastic deformation, tactile feedback signal, and the change data of cardiac function parameters) and the second feedback information (the change of UI element selection state). The system extracts key features from these feedbacks through information extraction algorithms, such as catheter operation accuracy, catheter path selection, drug selection and dosage, etc., to form feature vectors. Feature extraction uses multi-dimensional data dimensionality reduction technology to convert complex user operation information into quantifiable feature parameters.

[0066] After the feature vector is generated, through the knowledge point mapping algorithm, based on the pre-established knowledge point-feature correlation matrix, the correlation scores between the feature vector and each knowledge point are calculated. The knowledge point-feature correlation matrix is a two-dimensional table, where the rows represent different knowledge points and the columns represent various operation features. The matrix element values reflect the importance of the features to the knowledge points. For example, for the knowledge point of "heart anatomical structure", the weight of the catheter path selection feature is relatively high; while for the knowledge point of "drug mechanism of action", the weights of the drug selection and dosage features are greater. By calculating the weighted dot product of the feature vector and each row of the correlation matrix, the matching degree scores between the user's operations and each knowledge point are obtained.

[0067] After the matching degree scores are calculated, the system uses a threshold screening and sorting algorithm to determine the most relevant knowledge points. Specifically, first, a matching degree threshold is set, and all knowledge points with a matching degree exceeding this threshold are screened out. Then, they are sorted from high to low according to the matching degree, and the top N knowledge points are selected as the focus of this assessment. This dynamic matching mechanism ensures that the assessment content is closely related to the current operation focus of the learner, improving the pertinence of the evaluation.

[0068] After determining the target knowledge points, the system retrieves the corresponding questions from the assessment question group database and uses an adaptive algorithm for question screening. After retrieving the questions, the system presents the assessment question group to the learner through the AR interface. The assessment question group appears in the form of multiple-choice questions, with each question containing four options. The learner selects the answer through gesture interaction (such as direct clicking). The system records the learner's answering situation, including the selected answer, answering time, hesitation behavior during the answering process (such as switching between different options multiple times), etc., to form answering behavior data.

[0069] After the answering is completed, the system conducts a multi-dimensional evaluation of the answering results. First is the basic score, that is, the score is calculated according to the correctness of the answer. Each question is set with a basic score value, getting full marks for correct answers and zero marks for wrong answers. Secondly is the time weighting, which adjusts the basic score according to the answering speed. Answering too fast or too slow will both reduce the final score. Answering quickly may reflect that the learner has a solid grasp, but it may also be a guess; answering too slowly may indicate unclear understanding of the knowledge point. The time weighting is implemented through an inverted U-shaped function. A best answering time window is set, and answering within this window gets the highest weight, and the weight gets lower the farther away from it.

[0070] In addition to correctness and time factors, the system also considers the answering behavior characteristics for evaluation. For example, frequently switching options may indicate that the learner does not have a solid grasp of the knowledge point, and even if the answer is finally correct, the weight will be appropriately reduced in the scoring; on the contrary, directly selecting the correct answer indicates a solid grasp of the knowledge point and will get extra points.

[0071] After the scoring for each dimension is completed, the system calculates the comprehensive score through the weighted average method and compares it with the preset ability level division criteria to determine the mastery level of the learner on the current knowledge point. The mastery levels are divided into four levels: "proficient", "skilled", "basically mastered", and "needs improvement", and each level corresponds to a score range. The final evaluation results of the learner include the mastery levels of each knowledge point, detailed answer analysis, and targeted learning suggestions.

[0072] As Figure 2 shown, the present invention also provides an AR-based teaching evaluation system for animal cardiac catheterization intubation, including: A head-mounted glasses-type virtual reality all-in-one machine, which is used to identify the current scene where the user is located and collect environmental parameter information.

[0073] Among them, the head-mounted glasses-type virtual reality all-in-one machine includes: a binocular stereo vision camera, which is used to collect real environment images and calculate depth information; an attitude sensor group, which is used to monitor head movements and update the perspective information in real time; a supporting handle device, which is used to provide spatial positioning and support button input; a perspective display screen, which is used to fuse and display the real environment and virtual elements.

[0074] In addition, a simulation teaching unit is also provided in the head-mounted glasses-type virtual reality all-in-one machine, including: an augmented reality recognition sub-unit, which identifies the current scene where the user is located through the AR operation device lens and obtains the corresponding parameter information; an augmented reality model generation sub-unit, which generates a three-dimensional teaching model based on the parameter information of the current scene, fuses the real scene video and the three-dimensional model, and outputs and generates the final augmented reality model image; a mixed reality interaction sub-unit, which is used for the user to operate on the generated three-dimensional teaching model and the UI interface; a teaching sub-unit, which provides text annotations and corresponding voice broadcasts of the three-dimensional teaching model related to teaching knowledge points for the user in the AR scene.

[0075] In application, the mixed reality interaction sub-unit includes: three-dimensional model manipulation controls, which take, translate, zoom in, zoom out, or rotate the angle of the three-dimensional model by identifying different operation gestures of the user; select and confirm the UI button by identifying different operation gestures of the user; a mode switching control. The teaching software provides two forms of knowledge point presentation: the "three-dimensional model" mode and the "UI interface" mode, and the user can freely switch and select.

[0076] Furthermore, the UI interface is a two-dimensional component arranged in different planes. The same plane includes a title bar, an information bar, and UI buttons, including: a title bar, which presents the name of the item currently operated by the user; an information bar, which presents the detailed content of the item currently operated by the user in the form of text, pictures, and videos; a UI button. After the user makes an interactive selection, the current UI interface will jump to the new interface linked by the UI button.

[0077] A 3D model processing module for generating a virtual simulation teaching model based on environmental parameter information.

[0078] Among them, the 3D model processing module includes: a model library storage unit for storing a 3D heart model and an experimental model of catheter intubation surgery on domestic animals; a rendering engine for generating various display modes of solid images and transparent images; a physical collision engine for simulating the physical properties of the interaction between the catheter and the blood vessel; and a drug reaction simulator for calculating the influence effect of different drugs on cardiac function parameters.

[0079] A user interaction module for receiving user operation gestures and generating an interaction instruction set.

[0080] A user feedback module for evaluating the interaction operation result based on the interaction instruction set and generating a learning evaluation result.

[0081] Among them, the user feedback module includes: an examination question bank database for storing a set of multiple-choice questions corresponding to knowledge points; a scoring and analysis unit for generating evaluation data based on operation accuracy and answering situations; a learning progress tracker for recording the user's learning trajectory and knowledge mastery level; and a feedback report generator for outputting a learning report including short-term goals and long-term suggestions.

[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0083] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AR-based teaching evaluation method for animal cardiac catheterization, characterized in that, Including: S1: Collect user identity information, verify the user identity information, and obtain user authorized login data; S2: Identify the current scene where the user is located according to the user authorized login data, perform three-dimensional modeling on the scene parameter information, and obtain a virtual simulation teaching model; S3: Perform animal cardiac catheterization training operations through the virtual simulation teaching model, and at the same time identify and process the user's operation gestures to obtain user feedback information; S4: Match the corresponding knowledge point assessment question groups according to the user feedback information, and evaluate the answering results based on the knowledge point assessment question groups to obtain a learning assessment result.

2. The method for teaching evaluation of animal cardiac catheterization based on AR according to claim 1, wherein Step S2 further includes: S21: Scan the environment through the AR device camera, and perform spatial positioning processing on the obtained environmental information to obtain three-dimensional space coordinate data; S22: Call a preset model library, and fuse the preset model with the environmental real scene according to the three-dimensional space coordinate data to obtain a basic simulation teaching model; the basic simulation teaching model includes a three-dimensional heart model and an animal cardiac catheterization surgery experimental model; S23: Perform transparency adjustment processing on the virtual simulation teaching model through a deep learning algorithm to obtain a virtual simulation teaching model with the switching function between the entity display mode and the transparent display mode.

3. The method for teaching and evaluating animal cardiac catheterization based on AR according to claim 2, wherein, The obtaining steps of the three-dimensional heart model in step S22 include: S2211: Collect real heart anatomical data through a three-dimensional modeling tool, perform image reconstruction on the anatomical data, and obtain an initial three-dimensional heart model; S2212: Establish a cardiac cavity space topological structure according to the initial three-dimensional heart model, perform path planning on the cardiac blood vessel directions, and obtain blood flow dynamic simulation data; S2213: Perform cardiac beating simulation on the blood flow dynamic simulation data through a physics engine to obtain a three-dimensional heart model with physiological cycle changes.

4. The method for teaching evaluation of animal cardiac catheterization based on AR according to claim 2, wherein The obtaining steps of the animal cardiac catheterization surgery experimental model in step S22 include: S2221: Collect real operating table data and surgical instrument data through a three-dimensional scanning device, and perform mesh optimization on the operating table data and surgical instrument data to obtain an operating table three-dimensional model and a surgical instrument three-dimensional model respectively; S2222: Construct an animal body structure according to animal anatomy data, perform hierarchical processing on the animal internal organs, and obtain an animal three-dimensional model; S2223: Model the animal artery-heart blood vessel system through a computational fluid dynamics algorithm to obtain an animal cardiac catheterization surgery experimental model.

5. A method for teaching evaluation of animal cardiac catheterization based on AR according to claim 1, characterized in that, Step S3 further includes: S31: Perform animal cardiac catheterization training operations through the virtual simulation teaching model to obtain first feedback information; S32: Detect the user's finger movements through a gesture recognition algorithm, analyze the action types, and obtain gesture operation types; the gesture operation types include direct click, single-handed pinching, single-handed rotation, two-handed rotation, and two-handed zoom; S33: Trigger the corresponding model transformation function according to the gesture operation type, perform transformation matrix calculation processing on the virtual simulation teaching model, and obtain the transformation effect of the model; the transformation effects include translation effect, rotation effect, and zoom effect; S34: Determine and process the interaction between the user gesture and the UI interface through physical collision detection, obtain the second feedback information on the change in the selection state of the UI element, and output the first feedback information and the second feedback information as user feedback information.

6. The method for teaching evaluation of animal cardiac catheterization based on AR according to claim 5, wherein Step S31 specifically includes: Simulate the physical characteristics of a real catheter through a virtual catheter control algorithm, perform real-time calculation on the catheter deformation, and obtain the elastic deformation feedback; Perform collision detection processing on the catheter advancement path according to the vascular anatomical structure in the virtual simulation teaching model to obtain the tactile feedback signal; Perform physiological response calculation processing on different drug injections through a drug reaction simulation system to obtain the data on the change in cardiac function parameters.

7. An AR-based teaching evaluation system for animal cardiac catheterization intubation, characterized in that, Include: A head-mounted glasses-type virtual reality all-in-one machine, used to identify the current scene where the user is located and collect environmental parameter information; A three-dimensional model processing module, used to generate a virtual simulation teaching model according to the environmental parameter information; A user interaction module, used to receive the user operation gesture and generate an interaction instruction set; A user feedback module, used to evaluate the interaction operation result based on the interaction instruction set and generate a learning evaluation result.

8. The AR-based animal cardiac catheterization teaching and evaluation system according to claim 7, characterized in that, The head-mounted glasses-type virtual reality all-in-one machine includes: A binocular stereo vision camera, used to collect real environment images and calculate depth information; An attitude sensor group, used to monitor the head movement and update the perspective information in real time; A supporting handle device, used to provide spatial positioning and support button input; A perspective display screen, used to fuse and display the real environment and virtual elements.

9. The AR-based animal cardiac catheterization teaching and evaluation system according to claim 7, characterized in that, The three-dimensional model processing module includes: A model library storage unit, used to store the three-dimensional model of the heart and the experimental model of animal cardiac catheterization surgery; A rendering engine, used to generate various display modes of entity images and transparent images; A physical collision engine, used to simulate the physical characteristics of the interaction between the catheter and the blood vessel; A drug reaction simulator, used to calculate the influence effect of different drugs on cardiac function parameters.

10. The AR-based animal cardiac catheterization teaching and evaluation system according to claim 7, wherein, The user feedback module includes: An examination question bank database, used to store the multiple-choice question set corresponding to the knowledge points; A scoring and analysis unit, used to generate evaluation data according to the operation accuracy and the answering situation; A learning progress tracker, used to record the user's learning trajectory and knowledge mastery level; A feedback report generator, used to output a learning report including short-term goals and long-term suggestions.

Images