Multifunctional Surgical Auxiliary Teaching Platform

Abstract

The present invention relates to the field of medical education and surgical training technology, and discloses a multifunctional surgical auxiliary teaching platform, including: a data acquisition and feedback module, which is used to collect the force feedback data, position data and video data of trainees in surgical operations in real time through sensors, and provide tactile and visual feedback; an AI intelligent analysis and error recognition module, which is used to analyze the data collected by the data acquisition and feedback module in real time, identify errors in operations and provide corrective feedback. Through real-time data acquisition and feedback, the accuracy of trainees' surgical operations and error correction capabilities are improved; a reinforcement learning algorithm is used to dynamically adjust the training difficulty, provide personalized learning paths, help trainees focus on improving weak links and gradually master complex operations; support virtual operating rooms and remote collaboration, enhance teamwork training and adaptability, and provide a flexible learning experience.

Description

Technical Field

[0001] The present invention relates to the technical field of medical education and surgical training, and in particular to a multifunctional surgical auxiliary teaching platform. Background Art

[0002] With the continuous development of medical technology, the complexity and precision requirements of surgical operations are also increasing, and surgical operation training has become particularly important. Existing surgical teaching platforms mostly use traditional simulators, video teaching or surgical observation. Although these methods can help trainees to initially master the surgical process, they lack real-time feedback and personalized guidance during the training process, making it difficult to effectively improve trainees' practical operation skills. In addition, virtual reality (VR) technology and remote collaboration are gradually being applied to surgical training, but current technologies are mostly limited to basic scene simulations and cannot achieve high-precision data collection, real-time error correction, and complex teamwork training.

[0003] The construction of existing technologies is based on fixed teaching cases and surgical scenarios, and relies on simulators and basic virtual scenarios for surgical training. However, such platforms firstly lack a real-time operation feedback mechanism, and trainees cannot perceive errors in time during operation, resulting in weak error correction capabilities. Secondly, it is impossible to dynamically adjust the training difficulty according to the trainees' operation performance, and there is a lack of personalized teaching paths. Finally, the remote collaboration function is limited, making it difficult to achieve collaborative training of multiple trainees and real-time guidance from experts, which affects the trainees' teamwork training and their ability to cope with complex surgical scenarios. These problems restrict the effectiveness of existing surgical teaching platforms and cannot meet the high requirements of modern medical education for accuracy and flexibility. Summary of the invention

[0004] In view of the deficiencies of the prior art, the present invention provides a multifunctional surgical operation auxiliary teaching platform, which solves the problems of insufficient real-time feedback in surgical operation training, lack of personalized learning paths, and inability to effectively conduct remote collaboration and team training in the prior art.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: a multifunctional surgical operation auxiliary teaching platform, comprising:

[0006] The data acquisition and feedback module is used to collect the trainee's force feedback data, position data and video data during the surgical operation in real time through sensors, and provide tactile and visual feedback;

[0007] An AI intelligent analysis and error recognition module is used to perform real-time analysis on the data collected by the data collection and feedback module, identify errors in operation and provide corrective feedback;

[0008] Adaptive learning and personalized path generation module, used to dynamically adjust the training difficulty according to the trainees' operation performance and generate personalized learning paths;

[0009] Remote collaboration and virtual operating room module, used to support multi-user remote collaborative training and expert remote guidance;

[0010] The data accumulation and AI model continuous optimization module is used to optimize the AI ​​model through the accumulation of operational data and improve error recognition and learning feedback capabilities.

[0011] Preferably, the data collection and feedback module includes:

[0012] Force feedback sensor, used to detect the force applied by trainees during surgical operations. The sensor determines whether standard operations are met by providing real-time feedback on changes in instrument force;

[0013] Position sensors are used to monitor the position of surgical instruments in three-dimensional space and obtain trajectory information of trainees’ operations;

[0014] High-definition camera, used to obtain video data of trainees during surgery.

[0015] Preferably, the data collection and feedback module further includes a tactile feedback device for providing vibration feedback when force deviation is detected to remind the trainee to make adjustments.

[0016] Preferably, the AI ​​intelligent analysis and error recognition module includes:

[0017] A deep learning model, used to analyze the trainee's surgical operation data, wherein the deep learning model includes a convolutional neural network and a long short-term memory network;

[0018] The error recognition system is used to determine the type of error in the operation by analyzing the operation trajectory, force data and video data, and generate corrective feedback.

[0019] Preferably, the deep learning model performs feature extraction according to the following formula:

[0020] For video data I(t), the convolution feature I is calculated by the convolution kernel W conv , the formula is:

[0021] I conv (t)=W*I(t)

[0022] Among them, represents the convolution operation, I conv (t) is the feature map after convolution.

[0023] Preferably, the LSTM model is used to analyze time series data, using the following state update formula:

[0024] h t =σ(W h ·[xt ,h t-1 ]+b h )

[0025] Among them, x t The force feedback data and position data input of the students, h t is the hidden state at time step t.

[0026] Preferably, the adaptive learning and personalized path generation module includes a reinforcement learning algorithm, dynamically adjusts the training content according to the performance of the trainees, and optimizes learning through a reward mechanism, and the reward mechanism includes the following forms:

[0027]

[0028] where r t Indicates the reward or penalty value of the current operation.

[0029] Preferably, the remote collaboration and virtual operating room module supports virtual reality technology, trainees can perform operation simulations through the virtual operating room, the virtual surgical scene is constructed through 3D modeling technology, and multi-user collaborative training is allowed.

[0030] Preferably, the remote collaboration and virtual operating room module further supports real-time guidance from remote experts, who observe the trainee's operation process through a remote connection and provide voice or text guidance.

[0031] Preferably, the data accumulation and AI model continuous optimization module optimizes the error recognition capability and learning path by regularly retraining the AI ​​model, and performs dynamic model updates based on the accumulation of operational data.

[0032] The present invention provides a multifunctional surgical operation auxiliary teaching platform. It has the following beneficial effects:

[0033] 1. The present invention can collect all kinds of data of trainees in surgical operations in real time by integrating force feedback sensors, position sensors and high-definition cameras, and correct operational errors in time through intelligent feedback mechanisms. Through tactile feedback, visual prompts and real-time display of instrument operation trajectories, trainees can immediately perceive their own mistakes during operation and quickly adjust the operation mode, which greatly improves the accuracy of surgical operations and the efficiency of skill mastering. Trainees can gradually cultivate sensitivity to operational accuracy in simulation training, laying a solid foundation for reducing errors in clinical operations.

[0034] 2. The present invention adopts a reinforcement learning algorithm, which can dynamically adjust the training difficulty according to the real-time performance of the trainees and generate a personalized learning path. By analyzing the trainees' operation data, the system will automatically increase or decrease the difficulty of the training tasks to ensure that the training content matches the trainees' ability level. The personalized learning path design not only helps trainees focus on improving their weak links in a short period of time, but also gradually introduces more complex surgical situations according to their progress, so that trainees can steadily improve their surgical operation skills and gradually master more complex operations.

[0035] 3. The present invention creates a virtual operating room through virtual reality technology, and supports multi-student collaboration and remote expert guidance, thereby enhancing the training of surgical team collaboration capabilities. Students can simulate real surgical operation scenarios in a virtual environment and conduct teamwork training with other students. At the same time, remote experts can monitor operations in real time and provide guidance. This function not only expands the flexibility of teaching scenarios, allowing students to study anytime and anywhere, but also improves students' teamwork capabilities in actual operations and their ability to cope with complex situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a system architecture diagram of the present invention;

[0037] Figure 2 It is a structural schematic diagram of the data acquisition and feedback module of the present invention;

[0038] Figure 3 It is a structural diagram of the AI ​​intelligent analysis and error recognition module of the present invention;

[0039] Figure 4 It is a structural schematic diagram of the adaptive learning and personalized path generation module of the present invention;

[0040] Figure 5 It is a structural schematic diagram of the remote collaboration and virtual operating room module of the present invention;

[0041] Figure 6 This is a structural diagram of the data accumulation and AI model continuous optimization module of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Example:

[0044] Please see attached Figure 1 - Attachment Figure 6 , an embodiment of the present invention provides a multifunctional surgical auxiliary teaching platform, whose main function is to assist surgical training using advanced sensors, artificial intelligence (AI), virtual reality (VR) / augmented reality (AR) technology and remote collaboration functions. Through data acquisition, intelligent feedback, adaptive learning and remote collaboration, the present invention can monitor the trainees' operating performance in real time, provide personalized training feedback, and dynamically adjust the training content according to the trainees' learning progress. The working principle, implementation method and specific technical details of each module are described in detail below.

[0045] 1. Data collection and feedback module

[0046] 1.1 Data Collection Principle

[0047] One of the core modules of the present invention is the data acquisition and feedback module, which can obtain various operation data of trainees in real time during surgical operations by integrating multiple sensors, including force data, instrument position, hand movements and operation videos. The specific implementation is as follows:

[0048] Force feedback sensor: Installed in the operating part of the surgical simulator, it collects the force of the trainee's operation in real time. Each operating part, such as cutting knife, forceps, suture needle, etc., is integrated with a high-precision force feedback sensor, which can detect the change of the trainee's force at different operation stages. The force data is collected and converted into a three-dimensional force vector F(t) = [F x (t),F y (t),F z (t)], where F x (t), F y (t), F Z (t) is the force of each axis, and t is the operation time.

[0049] Position sensor: Through IMU (inertial measurement unit) and optical tracking technology, the position of surgical instruments in space is tracked in real time. The position data is expressed as three-dimensional coordinates p(t) = [x(t), y(t), z(t)]. The system can generate the complete operation trajectory of the instrument through these position data to ensure that the trainees' instrument operation complies with the standard path.

[0050] HD Camera: The platform is equipped with multiple HD cameras, which are installed at different viewing angles to capture the details of the trainees’ surgical operations. Video data (t) is collected in real time through high-frame rate cameras to provide support for subsequent AI visual analysis.

[0051] Multimodal data fusion: All collected data are aligned through time synchronization technology to ensure that force feedback, position and video data remain consistent on the timeline, forming a time-series multimodal operation data set for subsequent AI model analysis and feedback.

[0052] 1.2 Real-time feedback mechanism

[0053] After data collection is completed, the system helps trainees correct operational errors and improve operational accuracy in real time through a variety of feedback methods:

[0054] Haptic feedback: When the system detects a force deviation during operation, the force feedback device will respond immediately. For example, if the system detects that the force is too great (such as cutting too deep), the haptic feedback device will provide physical feedback to the trainee, which will appear as vibration of the surgical instrument or increased difficulty in operation. This feedback mechanism simulates the physical response of tissue to the instrument in real surgery, helping trainees perceive improper force.

[0055] Visual feedback: The system provides real-time visual prompts to trainees through screens or AR devices, such as deviation of the operation trajectory, force applied, etc. If the trainee's instrument operation deviates from the standard trajectory, the system will display the virtual path and the actual path at the same time to remind the trainee to adjust the operation position. This feedback mechanism is designed to help trainees correct their operations in a timely manner.

[0056] 2. AI intelligent analysis and error identification module

[0057] 2.1 Data preprocessing and feature extraction

[0058] The AI ​​intelligent analysis and error recognition module is the core of the platform, responsible for real-time analysis and error recognition of the collected multimodal data. The present invention adopts a model based on deep learning, including convolutional neural network (CNN) and long short-term memory network (LSTM), for feature extraction and analysis of video data and time series data.

[0059] Video data processing: The system uses a convolutional neural network (CNN) to analyze each frame of video I(t) collected during the surgery and extract the operating features of the instrument. CNN uses a series of convolutional layers and pooling layers to identify and track surgical instruments in the image. The formula for the convolution operation is:

[0060] I conv (t)=W*I(t)

[0061] Among them, I conv (t) is the image feature map after being processed by the convolution kernel W, which represents the convolution operation. Through the convolution layer, the system can extract the position, angle and operation status of the surgical instrument in the video.

[0062] Time series data processing: The force feedback data and instrument position data in the surgical operation are time series data. In order to capture the dynamic changes in the trainee's operation, the present invention uses a long short-term memory network (LSTM) to analyze the time series characteristics during the operation. LSTM updates the state through the following formula:

[0063] h t =σ(W h ·[x t ,h t-1 ]+b h )

[0064] Among them, x t is the student’s input for the current time step (including force feedback and position data), h t is the current hidden state, and σ is the activation function. LSTM detects errors or abnormal behaviors in the operation process by learning the students’ operation performance at different time steps.

[0065] 2.2 Error Identification and Feedback Mechanism

[0066] The present invention realizes automatic identification of operation errors through AI models and generates personalized feedback. The specific steps are as follows:

[0067] Error identification: The AI ​​model automatically identifies errors in operation by comparing the trainee's current operation data with the standard operation model. For example, the system can determine whether the suturing operation is too tight or the cutting depth is too deep by analyzing the force feedback data and position data. The CNN model is used to identify improper operation of surgical instruments, such as angle deviation; the LSTM model is used to identify discontinuous or too fast movements during the operation.

[0068] Feedback mechanism: After identifying an error, the system immediately generates feedback. For example, if it detects that the suture is too tight, the system will remind the trainee to reduce the force through visual and tactile feedback devices. For serious operating errors, such as cutting too deep, the system will provide strong tactile feedback to the trainee through a vibrating device and display a detailed error description on the screen.

[0069] 3. Adaptive learning and personalized path generation module

[0070] 3.1 Adaptive Learning Framework

[0071] In order to dynamically adjust training tasks and personalize learning paths, this paper uses a reinforcement learning (RL) algorithm. The RL algorithm continuously adjusts the difficulty and content of training based on the students' operational performance feedback (such as operational accuracy, completion time, etc.). Its core working principle is as follows:

[0072] State evaluation: Each student's operation performance is represented as a multidimensional state vector s t , including the trainee’s force feedback, machine position, operation time, and video analysis data. The RL algorithm evaluates the trainee’s current learning level based on their state.

[0073] Action selection: The system selects the next action in training (such as adjusting the difficulty of a task or adding a new operation task) through a reinforcement learning algorithm. Action set A t The options include "increase task complexity", "reduce task difficulty" and "maintain current difficulty". t Learn from learners’ performance and adjust tasks dynamically.

[0074] Reward mechanism: Each action of the student is accompanied by a reward or penalty value, the formula is as follows:

[0075]

[0076] where r t is the reward value of the current operation. If the student completes the task and operates correctly, the system gives a positive reward; if an incorrect operation occurs, a negative reward is given. By accumulating rewards, the system can gradually adjust the student's learning path.

[0077] 3.2 Personalized Path Generation

[0078] Based on the trainee's historical operation data and current performance, the system automatically generates a personalized learning path. The specific implementation steps are as follows:

[0079] Historical data analysis: The system will record each operation data of the students, including success rate, error type, operation time, etc. By analyzing this data, the AI ​​model can identify which operations the students perform well in and which operations have shortcomings.

[0080] Path optimization: Based on historical data, the system automatically adjusts future training tasks. For example, if a trainee performs poorly in a suturing operation, the system will automatically add more relevant training tasks for him or her and help him or her gradually master suturing skills by adjusting the difficulty of the tasks.

[0081] 4. Remote collaboration and virtual operating room module

[0082] 4.1 Virtual Operating Room

[0083] The virtual operating room module provides trainees with a realistic surgical training environment through virtual reality (VR) technology. The specific implementation methods are as follows:

[0084] 3D modeling and virtual environment generation: Through 3D modeling technology, the virtual operating room can simulate real surgical scenes, including operating tables, instruments, virtual patients, etc. Trainees interact with the virtual environment through VR equipment and operate virtual instruments to complete surgical training. The system collects virtual operation data in real time and performs intelligent analysis of the operation.

[0085] Use of virtual instruments: The instruments in the virtual operating room are consistent with real surgical instruments. When students operate virtual instruments in a virtual environment, the system will record their operation trajectory, force and movement details to ensure that students can simulate the real surgical operation process.

[0086] 4.2 Remote Collaboration

[0087] The present invention supports remote collaboration, allowing multiple trainees or experts to participate in training through the platform, thus enhancing team collaboration capabilities:

[0088] Remote expert guidance: Experts can enter the virtual operating room through remote video connection, observe the trainees' operations in real time, and provide voice or text guidance. Experts can view the trainees' operation data in real time through the system and give personalized feedback and suggestions.

[0089] Multi-student collaboration: Multiple students can enter the virtual operating room at the same time and divide the work to complete complex team surgical operations. The system will independently collect the operation data of each student, and independently analyze the performance of each student to evaluate their collaboration and operation skills.

[0090] 5. Data accumulation and AI model continuous optimization module

[0091] 5.1 Data accumulation and processing

[0092] The present invention continuously optimizes the feedback and learning capabilities of the AI ​​model through the long-term accumulation of student operation data. The specific implementation method is as follows:

[0093] Multimodal data storage: The system will upload the data of each operation to the cloud database to form the trainee's operation history file. The data includes visual information, force feedback, instrument trajectory, etc. After standardization, these data form a unified data set for retraining of AI models.

[0094] Data cleaning and standardization: The system removes noise and invalid data through data cleaning to ensure the high quality of data input into the AI ​​model and improve the training effect of the model.

[0095] 5.2 Model optimization and updating

[0096] By accumulating operational data, the system can regularly retrain and update the AI ​​model to maintain the model's advancement and accuracy:

[0097] Model retraining: Every time new operation data is accumulated, the system will retrain the AI ​​model, especially the CNN and LSTM models, to enable it to identify new error types and complex operation situations. Through continuous optimization, the system can adapt to different surgical scenarios and teaching needs.

[0098] Dynamic model update: The AI ​​model will be dynamically updated according to the actual operation needs of the students and the new medical standards. Whenever the system detects a new operation mode or technological advancement, the model will automatically adjust the parameters to ensure that the platform can adapt to the latest surgical teaching needs.

[0099] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Multifunctional surgical operation auxiliary teaching platform, characterized by: include: The data acquisition and feedback module is used to collect the trainee's force feedback data, position data and video data during the surgical operation in real time through sensors, and provide tactile and visual feedback; An AI intelligent analysis and error recognition module is used to perform real-time analysis on the data collected by the data collection and feedback module, identify errors in operation and provide corrective feedback; Adaptive learning and personalized path generation module, used to dynamically adjust the training difficulty according to the trainees' operation performance and generate personalized learning paths; Remote collaboration and virtual operating room module, used to support multi-user remote collaborative training and expert remote guidance; The data accumulation and AI model continuous optimization module is used to optimize the AI ​​model through the accumulation of operational data and improve error recognition and learning feedback capabilities.

2. The multifunctional surgical operation auxiliary teaching platform according to claim 1, characterized in that: The data collection and feedback module includes: Force feedback sensor, used to detect the force applied by trainees during surgical operations. The sensor determines whether standard operations are met by providing real-time feedback on changes in instrument force; Position sensors are used to monitor the position of surgical instruments in three-dimensional space and obtain trajectory information of trainees’ operations; High-definition camera, used to obtain video data of trainees during surgery.

3. The multifunctional surgical operation auxiliary teaching platform according to claim 2, characterized in that: The data collection and feedback module further includes a tactile feedback device for providing vibration feedback when force deviation is detected to remind the trainee to make adjustments.

4. The multifunctional surgical operation auxiliary teaching platform according to claim 1, characterized in that: The AI ​​intelligent analysis and error recognition module includes: A deep learning model, used to analyze the trainee's surgical operation data, wherein the deep learning model includes a convolutional neural network and a long short-term memory network; The error recognition system is used to determine the type of error in the operation by analyzing the operation trajectory, force data and video data, and generate corrective feedback.

5. The multifunctional surgical operation auxiliary teaching platform according to claim 4, characterized in that: The deep learning model performs feature extraction according to the following formula: For video data I(t), the convolution feature I is calculated by the convolution kernel W conv , the formula is: I conv (t)=W*I(t) Among them, represents the convolution operation, I conv (t) is the feature map after convolution.

6. The multifunctional surgical operation auxiliary teaching platform according to claim 4, characterized in that: The LSTM model is used to analyze time series data and uses the following state update formula: h t =σ(W h ·[x t ,h t-1 ]+b h ) Among them, x t The force feedback data and position data input of the students, h t is the hidden state at time step t.

7. The multifunctional surgical operation auxiliary teaching platform according to claim 1, characterized in that: The adaptive learning and personalized path generation module includes a reinforcement learning algorithm, dynamically adjusts the training content according to the student's performance, and optimizes learning through a reward mechanism, which includes the following forms: where r t Indicates the reward or penalty value of the current operation.

8. The multifunctional surgical operation auxiliary teaching platform according to claim 1, characterized in that: The remote collaboration and virtual operating room module supports virtual reality technology. Trainees can perform operation simulations in a virtual operating room. The virtual surgical scene is constructed through 3D modeling technology and allows multi-user collaborative training.

9. The multifunctional surgical operation auxiliary teaching platform according to claim 1, characterized in that: The remote collaboration and virtual operating room module further supports real-time guidance from remote experts, who observe the trainees' operation process through remote connections and provide voice or text guidance.

10. The multifunctional surgical operation auxiliary teaching platform according to claim 1, characterized in that: The data accumulation and AI model continuous optimization module optimizes the error recognition capability and learning path by regularly retraining the AI ​​model, and dynamically updates the model based on the accumulation of operational data.

Images