Interventional medical operation training system
By using an interventional medical procedure training system, which employs data gloves and multimodal deep learning models to assess students' skills and provide personalized training courses, the system solves the problem of low learning efficiency in traditional training methods and achieves highly efficient interventional medical procedure training.
Patent Information
- Application Number
- CN202511861259.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-02-24
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional interventional medical procedure training methods rely on hands-on teaching between mentors and apprentices. Limited by time, space, and expert resources, existing simulation training equipment lacks realism and interactivity, resulting in a steep learning curve and low efficiency for trainees, and failing to adapt to different training scenarios or provide personalized guidance.
An interventional medical operation training system was designed, including a first data glove, audio and video acquisition equipment, a central processing and conversion module, and a scene presentation device. By capturing the teacher's action and voice data, image and audio data are generated to assess students' operational skills, and personalized training courses are provided using a multimodal deep learning model and a reinforcement learning recommendation module.
It enables efficient and personalized interventional medical procedure training, improves trainees' learning outcomes and training efficiency, and provides an immersive teaching experience and precise skills assessment and guidance.
Smart Images
Figure CN121565035A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent medical device technology, and in particular to an interventional medical operation training system. Background Technology
[0002] Interventional medical procedures are critical skills in surgery, anesthesiology, pain management, and emergency medicine, requiring operators to possess high precision in hand movements and coordination. Traditional training methods primarily rely on hands-on instruction between master and apprentice, but are limited by time, space, sterile environments, and insufficient expert resources. Existing simulation training equipment often lacks realism and interactivity, failing to fully simulate the subtle movements and tactile feedback of actual procedures, resulting in a steep learning curve and low efficiency for trainees. Furthermore, existing assistive operating systems have fixed functions and cannot adapt to different training scenarios or provide data-driven personalized guidance.
[0003] Therefore, improving trainees' learning outcomes in interventional medical procedures is a pressing technical problem that needs to be addressed. Summary of the Invention
[0004] In view of the above problems, the present invention provides an interventional medical operation training system that overcomes or at least partially solves the above problems.
[0005] In a first aspect, the present invention provides an interventional medical operation training system, comprising: The first data glove, worn on the teacher's hand, is used to capture the teacher's first hand movements, first hand force, and first hand posture information during interventional medical procedures. Audio and video capture devices, worn on the teacher's head, are used to capture images and audio of the teacher during interventional medical procedures. The central processing and conversion module, connected to the first data glove and the audio and video acquisition device, is used to generate image data and audio data based on images and audio during interventional medical procedures. A scene presentation device, connected to the central processing and conversion module, is used to present the image data and audio data; The second data glove, connected to the central processing and conversion module, is worn on the student's hand and is used to collect information on the student's second hand movements, second hand strength, and second hand posture during the student's autonomous execution of the interventional medical procedure. The central processing and conversion module is also used to evaluate the student's operational skills based on the second hand movement, second hand force, second hand posture information, first hand movement, first hand force, and first hand posture information, and obtain the evaluation results.
[0006] Preferably, the central processing conversion module is further configured to: Based on the first hand movement, first hand force, and first hand posture information, a digital twin mechanism is used to generate hand data; The second data glove includes: a bionic multi-finger manipulation device, and is further used for: Based on the hand data, the student is guided by the bionic multi-finger manipulation device to feel the teacher's first hand movement, first hand force, and first hand posture data during the interventional medical procedure.
[0007] Preferably, the scene presentation device is any one of the following: Two-dimensional display devices, virtual reality devices, augmented reality devices, and mixed reality devices.
[0008] Preferably, the interventional medical procedures include: vascular puncture, nerve block, spinal canal puncture, catheter intervention, puncture injection, and puncture drainage.
[0009] Preferably, the audio and video acquisition device specifically includes: Microphone, image acquisition device, wherein the image acquisition device includes: a single camera, two cameras, or two or more cameras; The microphone is used to collect the teacher's voice during interventional medical procedures; The image acquisition device is used to acquire images of the teacher performing interventional medical procedures from one or more perspectives.
[0010] Preferably, the central processing conversion module is further configured to: A multimodal deep learning model is used to determine the skill representation vector of the student in the process of performing interventional medical procedures based on the second hand movement, second hand force and second hand posture information. The evaluation result is determined by comparing the skill representation vector with the target skill representation vector, where the target skill vector is extracted by the teacher during the execution of interventional medical procedures.
[0011] Preferably, the multimodal deep learning model is trained based on historical hand movements, hand strength, hand posture information, and historical evaluation results of students in the course of historical interventional medical procedures, and is used to evaluate students' operational skills in performing interventional medical procedures.
[0012] Preferably, it further includes: a reinforcement learning recommendation module, used for: Based on the assessment results, training courses tailored to the assessment results are provided to bridge the students' skill gaps.
[0013] Preferably, both the first data glove and the second data glove include: Gloves, worn over the hands; An inertial measurement unit is installed on the glove, with corresponding locations on the back of the hand and wrist area, to capture hand movement trajectories and spatial positioning; A flexible bending sensor is installed on the glove, with corresponding distributions at each finger joint, to detect the bending angle of the hand; Miniature pressure sensors are mounted on the glove and distributed at the fingertips to sense tactile pressure.
[0014] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: This invention provides an interventional medical procedure training system, comprising: a first data glove worn on the teacher's hand to capture the teacher's first hand movements, first hand force, and first hand posture information during interventional medical procedures; an audio / video acquisition device worn on the teacher's head to capture images and audio during the interventional medical procedure; a central processing and conversion module connected to the first data glove and the audio / video acquisition device to generate image data and audio data based on the images and audio during the interventional medical procedure; a scene presentation device connected to the central processing and conversion module to present the image data and audio data; and a second data glove connected to the central processing and conversion module and worn on the student's hand to capture the student's second hand movements, second hand force, and second hand posture information during the student's independent performance of interventional medical procedures. The central processing and conversion module is also used to evaluate the student's operational skills based on the second hand movements, second hand force, second hand posture information, and first hand movements, first hand force, and first hand posture information, obtaining evaluation results, thereby effectively improving the training effect of interventional medical procedures with the help of the data gloves. Attached Figure Description
[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic diagram of the structure of the interventional medical operation training system in an embodiment of the present invention is shown. Detailed Implementation
[0016] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.
[0017] Example 1: Embodiments of the present invention provide an interventional medical procedure training system, such as... Figure 1 As shown, it includes: The first data glove 101 is worn on the teacher's hand and is used to capture the teacher's first hand movements, first hand force, and first hand posture information during interventional medical procedures. Audio and video acquisition device 102, worn on the teacher's head, is used to acquire images and voice recordings of the teacher during interventional medical procedures. The central processing and conversion module 103 is connected to the first data glove and the audio and video acquisition device, and is used to generate image data and audio data based on the images and audio during the interventional medical operation. The scene presentation device 104 is connected to the central processing and conversion module 103 and is used to present image data and audio data. The second data glove 105 is connected to the central processing and conversion module 103 and is worn on the student's hand. It is used to collect information on the student's second hand movements, second hand strength, and second hand posture during the student's independent execution of interventional medical procedures. The central processing and conversion module 103 is also used to evaluate the student's operational skills based on the second hand movement, second hand force, second hand posture information, first hand movement, first hand force, and first hand posture information, and obtain the evaluation results.
[0018] This invention constructs a complete closed-loop system of "perception-transmission-reproduction-evaluation," mainly composed of three parts: teacher-side equipment, student-side equipment, and a central processing and conversion server. The teacher-side equipment uses a first data glove 101 to capture the teacher's first hand movements, force, and posture in real time, and simultaneously captures images and audio from the teacher during the interventional medical procedure using an audio / video acquisition device 102. The student-side equipment uses a second data glove 105 to capture the student's second hand movements, force, and posture during the interventional medical procedure, and displays the images through a scene presentation device 104. Finally, the central processing and conversion module 103 evaluates the student's operational skills and obtains the evaluation results.
[0019] The following sections will describe each component in detail.
[0020] The teacher-side equipment includes a first data glove 101 and an audio / video acquisition device 102, both of which are worn by the teacher. The first data glove 101 is worn on the teacher's hands, and the audio / video acquisition device 102 is worn on the teacher's head.
[0021] The first data glove 101 includes: Gloves, worn over the hands; An inertial measurement unit (IMU) is installed on the glove, with corresponding locations on the back of the hand and wrist area. It is used to capture the hand's movement trajectory and spatial positioning, which is equivalent to the first hand movement. A flexible bending sensor is installed on the glove, with corresponding distributions at each finger joint, to detect the bending angle of the hand, which is equivalent to the first hand posture information; Miniature pressure sensors are installed on the glove, with corresponding distributions at the fingertips, to sense tactile pressure, equivalent to the force applied by the hand.
[0022] When wearing the first data glove 101, a sterile surgical glove can be worn over it to ensure that the system can be used in both simulated environments and strict infection control environments such as real operating rooms, thus breaking down the barrier between high-precision sensing devices and clinical aseptic requirements.
[0023] The various sensors described above capture information about the teacher's first hand movements, the force applied to the first hand, and the posture of the first hand during interventional medical procedures.
[0024] At the same time, the audio and video acquisition device 102 also captures images and voice recordings of the teacher during the interventional medical procedure.
[0025] The audio and video acquisition device 102 includes: Microphone, image acquisition device, wherein the image acquisition device includes: a single camera, two cameras, or two or more cameras; The microphone is used to collect the teacher's voice during the interventional medical procedure; the image acquisition device is used to collect images of the teacher during the interventional medical procedure from one or more perspectives.
[0026] Because the interventional medical procedures performed by teachers have a certain degree of guidance, their images and audio have a guiding role.
[0027] The interventional medical procedure can be any of the following: Vascular puncture, nerve block, spinal canal puncture, catheter intervention, puncture injection, and puncture drainage. Of course, other types of interventional medical procedures are also included, but will not be detailed here.
[0028] Next, the first hand movements, first hand force, and first hand posture information captured by the first data glove 101 during the interventional medical procedure, as well as the images and voices captured by the audio and video acquisition device 102 during the interventional medical procedure, are all transmitted to the central processing and conversion module 103.
[0029] The central processing and conversion module 103 generates image data and audio data based on the images and audio during the interventional medical procedure.
[0030] Next, the student-end device is accessed, which includes a scene presentation device 104 and a second data glove 105. The scene presentation device 104 is connected to the central processing and conversion module 103 and is used to present image and audio data. The second data glove 105 is also connected to the central processing and conversion module 103 and is worn on the student's hand. It is used to collect information on the student's second hand movements, second hand strength, and second hand posture during the student's autonomous performance of interventional medical procedures.
[0031] Specifically, the scene presentation device 104 can be any of the following: Two-dimensional display devices, virtual reality (VR) devices, augmented reality (AR) devices, and mixed reality (MR) devices. Specifically, the two-dimensional display device can be a flat display device or a curved display device; no limitation is made here. When the scene presentation device 104 is any one of a virtual reality device, augmented reality device, or mixed reality device, it can be worn on the student's head.
[0032] The scene presentation device 104 can present the teacher's first-person perspective video, giving students an immersive visual experience in the scene. At the same time, the audio device outputs the teacher's voice, enhancing the sense of presence.
[0033] In this scenario, students practice this interventional medical procedure independently, and the second data glove 105 collects information on the students' second hand movements, second hand strength, and second hand posture.
[0034] Next, these data are transmitted to the central processing and conversion module 103, which then evaluates the students' operational skills based on this data.
[0035] Specifically, the central processing and conversion module 103 is used for: A multimodal deep learning model is used to determine the skill representation vector of students in the process of performing interventional medical procedures based on the second hand movement, second hand force and second hand posture information. The evaluation results are determined by comparing the skill representation vector with the target skill representation vector, which is extracted by the teacher during the execution of interventional medical procedures.
[0036] The multimodal deep learning model is trained based on historical hand movements, hand strength, and hand posture information of students during historical interventional medical procedures, as well as historical evaluation results. It is used to assess students' operational skills in performing interventional medical procedures. These historical evaluation results can be human assessments conducted by teachers or experts on students during practical training.
[0037] Specifically, a large amount of historical hand movements, hand strength, and hand posture information, along with historical evaluation results, from students performing historical interventional medical procedures are input into the model for training, thereby obtaining a multimodal deep learning model. This multimodal deep learning model is used to evaluate students' operational skills in performing interventional medical procedures.
[0038] This multimodal deep learning module is used to obtain the target skill representation vector for teachers' operational skills in performing interventional medical procedures.
[0039] Next, the student's second hand movement, second hand force, and second hand posture information are input into the multimodal deep learning model to obtain the student's skill representation vector.
[0040] Next, the skill representation vector is compared with the target skill representation vector to obtain the evaluation result, which includes the student's skill gap.
[0041] Therefore, the system also includes: a reinforcement learning recommendation module, used for: Based on the assessment results, training courses tailored to the assessment findings will be provided to bridge the students' skills gaps.
[0042] For example, to address skill gaps such as inconsistent needle insertion angle control, specific tutorials that include breakdowns and exercises of the movement are recommended. This targeted training approach, enabling personalized training, can rapidly improve training efficiency.
[0043] The above describes the process of assisting students in training. Of course, the second data glove 105 can also be used to guide students to experience the teacher's operation process and enable them to quickly master the operation skills.
[0044] Specifically, the central processing and conversion module 103 is used for: Hand data is generated based on the first hand movement, the first hand force, and the first hand posture information.
[0045] The second data glove 105 also includes: a bionic multi-finger manipulation device, used for: Based on hand data, students are guided to use a bionic multi-finger manipulation device to feel the teacher's first hand movement, first hand force, and first hand posture data during interventional medical procedures.
[0046] This bionic multi-finger manipulation device allows control of two, three, four, or more fingers, enabling everything from minute finger manipulations (such as pinching a puncture needle) to coordinated overall movement (grasping an ultrasound probe and adjusting its angle). Its ergonomic bionic replication provides a realistic tactile experience and creates a highly immersive teaching environment. This allows students learning interventional medical procedures to feel the instructor's pressure and rhythm, receiving hands-on teaching and significantly improving learning efficiency and operational intuition. Furthermore, it can be used to target specific skill gaps for individual training.
[0047] By adopting the above training methods, students' training effectiveness can be improved quickly.
[0048] One or more technical solutions in the embodiments of the present invention have at least the following technical effects or advantages: This invention provides an interventional medical procedure training system, comprising: a first data glove worn on the teacher's hand to capture the teacher's first hand movements, first hand force, and first hand posture information during interventional medical procedures; an audio / video acquisition device worn on the teacher's head to capture images and audio during the interventional medical procedure; a central processing and conversion module connected to the first data glove and the audio / video acquisition device to generate image data and audio data based on the images and audio during the interventional medical procedure; a scene presentation device connected to the central processing and conversion module and worn on the student's head to present the image data and audio data; and a second data glove connected to the central processing and conversion module and worn on the student's hand to capture the student's second hand movements, second hand force, and second hand posture information during the student's independent performance of interventional medical procedures. The central processing and conversion module is also used to evaluate the student's operational skills based on the second hand movements, second hand force, second hand posture information, and first hand movements, first hand force, and first hand posture information, obtaining evaluation results, thereby effectively improving the training effect of interventional medical procedures with the help of the data gloves.
[0049] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.
[0050] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. An interventional medical procedure training system, characterized in that, include: The first data glove, worn on the teacher's hand, is used to capture the teacher's first hand movements, first hand force, and first hand posture information during interventional medical procedures. Audio and video capture devices, worn on the teacher's head, are used to capture images and audio of the teacher during interventional medical procedures. The central processing and conversion module, connected to the first data glove and the audio and video acquisition device, is used to generate image data and audio data based on images and audio during interventional medical procedures. A scene presentation device, connected to the central processing and conversion module, is used to present the image data and audio data; The second data glove, connected to the central processing and conversion module, is worn on the student's hand and is used to collect information on the student's second hand movements, second hand strength, and second hand posture during the student's autonomous execution of the interventional medical procedure. The central processing and conversion module is also used to evaluate the student's operational skills based on the second hand movement, second hand force, second hand posture information, first hand movement, first hand force, and first hand posture information, and obtain the evaluation results.
2. The system as described in claim 1, characterized in that, The central processing and conversion module is also used for: Based on the first hand movement, first hand force, and first hand posture information, a digital twin mechanism is used to generate hand data; The second data glove includes: a bionic multi-finger manipulation device, and is further used for: Based on the hand data, the student is guided by the bionic multi-finger manipulation device to feel the teacher's first hand movement, first hand force, and first hand posture data during the interventional medical procedure.
3. The system as described in claim 1, characterized in that, The scene presentation device is specifically any one of the following: Two-dimensional display devices, virtual reality devices, augmented reality devices, and mixed reality devices.
4. The system as described in claim 1, characterized in that, The interventional medical procedures include: vascular puncture, nerve block, spinal canal puncture, catheter intervention, puncture injection, and puncture drainage.
5. The system as described in claim 1, characterized in that, The audio and video acquisition device specifically includes: A microphone and an image acquisition device, wherein the image acquisition device includes: a single camera, two cameras, or two or more cameras; The microphone is used to collect the teacher's voice during interventional medical procedures; The image acquisition device is used to acquire images of the teacher performing interventional medical procedures from one or more perspectives.
6. The system as described in claim 1, characterized in that, The central processing and conversion module is used for: A multimodal deep learning model is used to determine the skill representation vector of the student in the process of performing interventional medical procedures based on the second hand movement, second hand force and second hand posture information. The evaluation result is determined by comparing the skill representation vector with the target skill representation vector, where the target skill representation vector is extracted by the teacher during the execution of interventional medical procedures.
7. The system as described in claim 6, characterized in that, The multimodal deep learning model is trained based on historical hand movements, hand strength, hand posture information, and historical evaluation results of students in the course of historical interventional medical procedures. It is used to evaluate students' operational skills in performing interventional medical procedures.
8. The system as described in claim 1, characterized in that, Also includes: The reinforcement learning recommendation module is used for: Based on the assessment results, training courses tailored to the assessment results are provided to bridge the students' skill gaps.
9. The system as described in claim 1, characterized in that, Both the first data glove and the second data glove include: Gloves, worn over the hands; An inertial measurement unit is installed on the glove, with corresponding locations on the back of the hand and wrist area, to capture hand movement trajectories and spatial positioning; A flexible bending sensor is installed on the glove, with corresponding distributions at each finger joint, to detect the bending angle of the hand; Miniature pressure sensors are mounted on the glove and distributed at the fingertips to sense tactile pressure.