Modularized VR / AR operation teaching system and method based on local large model

By using a modular VR/AR surgical teaching system driven by a local large model, combined with lightweight hardware and multimodal assessment, the system achieves adaptive generation and secure inference of teaching content, solves the dynamic adaptability and privacy issues of existing systems, provides high-fidelity multi-sensory interaction and accurate assessment, and improves the effectiveness of surgical training.

CN122090684APending Publication Date: 2026-05-26TONGJI UNIV
0 Cites 0 Cited by

Patent Information

Application Number
CN202610341326.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-03-19
Publication Date
2026-05-26

AI Technical Summary

Technical Problem

Existing VR/AR surgical teaching systems lack dynamic adaptability, cannot adjust training difficulty according to the student's level, have risks of network latency and privacy leakage, lack comprehensive collection and analysis of multimodal data, cannot provide accurate feedback, and are bulky with unrealistic force feedback.

Method used

A modular VR/AR surgical teaching system driven by a local large model is adopted. It combines lightweight hardware with multimodal fusion evaluation algorithms to achieve adaptive content generation and secure reasoning. The system deploys a large model through edge computing, integrates multi-dimensional perception units and multi-sensory feedback, and performs real-time error correction and quantitative evaluation.

Benefits of technology

It achieves automated and dynamic adaptation of teaching content, reduces customization costs, ensures the relevance and security of teaching, provides high-fidelity multi-sensory interaction and accurate quantitative assessment, solves the latency and privacy issues of traditional systems, and improves training effectiveness.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a modular VR / AR operation teaching system and method based on a local large model, and the system comprises a local large model training module which is disposed in a local computing environment, stores structured medical knowledge, and dynamically generates a teaching script and an examination question bank; the lightweight VR / AR hardware group is used for collecting multi-dimensional operation data of a user and providing multi-sensory feedback; the whole-process training control module is internally provided with a physical simulation engine and evaluation logic, constructs a virtual surgery scene, simulates tissue mechanical changes in real time, compares user operation with a standard surgery model, generates error correction guidance and generates a quantitative capability evaluation report based on whole-process data; according to the multi-scene data mapping interface, a modularized framework based on a standardized intermediate communication protocol is adopted, and low-coupling logic isolation and dynamic integration among functional modules are achieved through a unified data exchange format and abstract interface specifications. According to the method, an edge side deployment strategy is adopted, and patient privacy and VR / AR scene real-time force feedback and tissue deformation rendering are guaranteed.
Need to check novelty before this filing date? Find Prior Art