Bronchoscope simulation training system and method based on VR technology

Abstract

The invention provides a bronchoscope simulation training system and method based on a VR technology, and the system comprises a VR helmet which is used for generating a 3D simulation bronchoscope examination scene; the simulation operation handle is used for simulating the operation of the bronchoscope; the position sensor is used for capturing action data when the user operates the simulation operation handle; the controller is used for calculating and rendering an operation view angle and an operation object of a user in a 3D simulated bronchoscopy scene according to the received action data, and feeding back an operation torque through the simulated operation handle so as to form various operation experiences; and the adaptability training algorithm chip is used for performing proficiency evaluation according to the action data to obtain an operation proficiency, and dynamically adjusting the difficulty of the 3D simulated bronchoscopy scene according to the operation proficiency. According to the invention, a vivid virtual training environment is provided, so that a novice doctor can carry out simulation training under a non-invasive condition, and the operation skill is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of model design, in particular to a bronchoscope simulation training system and method based on VR technology. BACKGROUND

[0002] With the continuous development of medical technology, bronchoscopy has become an important means for the diagnosis of respiratory diseases. However, due to the high technical requirements and difficult operation, novice doctors often need a long time and a large amount of practical operation to master it. In addition, bronchoscopy has a certain degree of trauma, and novice doctors may cause additional pain to patients due to lack of experience in actual operation.

[0003] Therefore, a bronchoscope simulation training system based on VR technology is developed to solve the above problems. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present application is to provide a bronchoscope simulation training system and method based on VR technology.

[0005] To achieve the above purpose, the present application provides the following scheme: A bronchoscope simulation training system based on VR technology, comprising: a VR helmet for generating a 3D simulated bronchoscopy scene; a simulated operation handle for simulating the operation of a bronchoscope; a position sensor for capturing motion data when a user operates the simulated operation handle; a controller for calculating and rendering the user's operation perspective and operation object in the 3D simulated bronchoscopy scene according to the received motion data, and feeding back the operation torque through the simulated operation handle to form various operation experiences; an adaptive training algorithm chip for evaluating the operation proficiency according to the motion data, obtaining the operation proficiency, and dynamically adjusting the difficulty of the 3D simulated bronchoscopy scene according to the operation proficiency.

[0006] Preferably, it further comprises: a first sensory feedback module for playing real device operation sounds at the VR helmet according to the user's motion data.

[0007] Preferably, it further comprises: a second sensory feedback module for spraying liquid simulated lubricant at the VR helmet according to the user's motion data to simulate the disinfection operation after the completion of bronchoscopy.

[0008] Preferably, it further comprises: An intelligent analysis module is configured to monitor the action data in real time, compare the monitoring result with preset correct operation data, and provide visual guidance feedback information in the 3D simulated bronchoscopy scene if the comparison result is incorrect.

[0009] Preferably, further comprising: A storage module is configured to store the 3D simulated bronchoscopy scene and user data.

[0010] Preferably, further comprising: A network module is configured to download new 3D simulated bronchoscopy scenes and update software from the cloud.

[0011] Preferably, further comprising: A tutorial module is configured to input preset 3D simulated bronchoscopy scenes into the VR helmet for visual display.

[0012] A bronchoscopy simulation training method based on VR technology, comprising: Generating a 3D simulated bronchoscopy scene; Simulating the operation of a bronchoscope; Capturing action data when a user operates a simulated operation handle; According to the received action data, calculating and rendering the operation perspective and operation object of the user in the 3D simulated bronchoscopy scene, and feeding back the operation torque through the simulated operation handle to form various operation experiences; According to the action data, evaluating the proficiency, obtaining the operation proficiency, and dynamically adjusting the difficulty of the 3D simulated bronchoscopy scene according to the operation proficiency.

[0013] According to the specific embodiments of the present application, the following technical effects are provided: The present application provides a bronchoscopy simulation training system and method based on VR technology. The system comprises: a VR helmet configured to generate a 3D simulated bronchoscopy scene; a simulated operation handle configured to simulate the operation of a bronchoscope; a position sensor configured to capture action data when a user operates the simulated operation handle; a controller configured to calculate and render the operation perspective and operation object of the user in the 3D simulated bronchoscopy scene according to the received action data, and feed back the operation torque through the simulated operation handle to form various operation experiences; and an adaptive training algorithm chip configured to evaluate the proficiency according to the action data, obtain the operation proficiency, and dynamically adjust the difficulty of the 3D simulated bronchoscopy scene according to the operation proficiency. The present application provides a realistic virtual training environment, allowing novice doctors to perform simulation training without causing harm and improving their operation skills. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0015] Figure 1 The system structure schematic diagram provided by the embodiment of the present application is shown in DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0017] The purpose of the present application is to provide a bronchoscope simulation training system based on VR technology, which can provide a realistic virtual training environment, so that novice doctors can perform simulation training without injury and improve operation skills.

[0018] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0019] Figure 1 The system structure schematic diagram provided by the embodiment of the present application is shown in Figure 1 The present application provides a bronchoscope simulation training system based on VR technology, which includes: VR helmet, for generating a 3D simulated bronchoscopy scene; Simulation operation handle, for simulating the operation of bronchoscope; Position sensor, for capturing action data when the user operates the simulation operation handle; Controller, for calculating and rendering the operation perspective and operation object of the user in the 3D simulated bronchoscopy scene according to the received action data, and feeding back operation torque through the simulation operation handle to form various operation experiences; Adaptive training algorithm chip, for evaluating the operation proficiency according to the action data, obtaining the operation proficiency, and dynamically adjusting the difficulty of the 3D simulated bronchoscopy scene according to the operation proficiency.

[0020] Specifically, the embodiment adopts an adaptive training algorithm that dynamically adjusts the difficulty of the virtual training scenario based on the user's proficiency, ensuring personalized and optimized training results. The embodiment also analyzes the user's proficiency through intelligent algorithms and adjusts the difficulty and complexity of the virtual training scenario accordingly. The algorithm can also provide personalized learning suggestions and guidance based on the user's learning progress.

[0021] Demonstratively, the adaptive training algorithm of the embodiment is a dynamic difficulty adjustment algorithm, which dynamically adjusts the training difficulty based on the user's performance to achieve a personalized and optimized learning path. The working principle is as follows: Performance evaluation: The algorithm first evaluates the user's performance in the simulation training, including the accuracy of the operation, the speed, and the type of errors in the operation process, etc.

[0022] Difficulty adjustment: Based on the results of performance evaluation, the algorithm determines whether to adjust the difficulty of training through predetermined rules or machine learning models (such as generative adversarial networks). For example, if the user performs well for several times in a row, the system will increase the difficulty; conversely, if the user makes mistakes continuously, the system will reduce the difficulty.

[0023] Personalized training: The algorithm can also adjust the training content based on the user's long-term performance to ensure that each user can learn in an environment suitable for their own level.

[0024] Specifically, the intelligent algorithm in the embodiment is a real-time error correction and guidance algorithm, which aims to provide feedback and guidance in real time by analyzing the user's operation data, helping users correct errors and learn the correct operation method. The working principle is as follows: Real-time monitoring: The algorithm continuously monitors the user's operation data, including operation path, speed, force, etc.

[0025] Error detection and analysis: Using a pre-set correct operation model or a model trained by machine learning (such as RNN convolutional network), the algorithm can identify deviations or errors in the user's operation.

[0026] Immediate feedback: Once an error is detected, the algorithm immediately provides feedback and guidance to the user through various means such as vision, hearing, or touch, pointing out the error and suggesting improvement methods.

[0027] Specifically, the controller of the embodiment first performs the step of collecting data, which aims to obtain the user's action data and the input data of the operation handle, specifically including: Use of motion capture devices: Use motion capture devices (such as Vicon, OptiTrack, etc.) or VR controllers (such as HTC Vive, Oculus Rift, etc.) to capture the user's head and hand movements.

[0028] Controller handle input: Acquire input data of the controller handle (such as position, angle, button press, etc.).

[0029] Further, the controller of the embodiment performs a data processing step, which aims to process and analyze the collected motion data and handle input data, specifically including: Motion data analysis: Analyze the data from the motion capture device or VR controller into the user's head position, viewing direction, and hand position and posture.

[0030] Handle input analysis: Analyze the input data of the controller handle into specific control instructions, such as movement, rotation, button press, etc.

[0031] Further, the controller of the embodiment performs a view angle calculation step, which aims to calculate the user's view angle in the 3D scene based on the user's head position and viewing direction, specifically including: Head position and direction: Calculate the user's view angle position and direction in the 3D scene based on the collected head position and direction data.

[0032] View angle transformation matrix: Generate a view angle transformation matrix for adjusting the view angle in 3D rendering.

[0033] Further, the controller of the embodiment performs an operation object calculation step, which aims to determine the object currently operated by the user based on the hand position and posture, specifically including: Hand position and posture: Determine the hand position and direction in the 3D scene based on the collected hand position and posture data.

[0034] Collision detection: Perform collision detection in the 3D scene to determine whether the hand collides with an object, thereby determining the currently operated object.

[0035] Further, the controller of the embodiment performs a 3D rendering step, which aims to render a 3D simulated bronchoscopy scene, displaying the user's view angle and operation object, specifically including: Scene construction: Use 3D modeling software (such as Blender, Maya, etc.) to construct a scene model of bronchoscopy.

[0036] Rendering engine: Use a 3D rendering engine (such as Unity, Unreal Engine, etc.) to render the scene, applying the view angle transformation matrix and the state of the operation object.

[0037] Real-time rendering: Real-time update of the rendered image to reflect the changes in the user's view angle and the state of the operation object.

[0038] Further, the controller of the present embodiment again performs the force feedback calculation step, which aims to calculate the torque that needs to be fed back according to the user's operation, and feeds it back to the user through the operation handle, specifically including: Operation torque calculation: Calculate the torque that needs to be fed back according to the user's operation object and operation force.

[0039] Force feedback algorithm: Use a force feedback algorithm (such as a PID controller) to calculate the specific torque value.

[0040] Handle force feedback: Feedback the calculated torque to the user through the force feedback device of the operation handle (such as a vibration motor, force feedback motor, etc.).

[0041] Further, the controller of the present embodiment again performs the comprehensive experience step, which aims to form the user's various operation experiences in the 3D simulated bronchoscopy scene, specifically including: View and operation synchronization: Ensure real-time synchronization of the user's view and operation object in the 3D scene.

[0042] Force feedback experience: Provide corresponding force feedback experience according to different operation scenarios and operation force.

[0043] Interaction experience optimization: Continuously optimize the user's interaction experience to ensure smoothness and realism of operation.

[0044] The tools and techniques used in the controller in the present embodiment are as follows: Motion capture equipment and VR controller: Vicon, OptiTrack, HTC Vive, Oculus Rift, etc.

[0045] 3D modeling software: Blender, Maya, etc.

[0046] 3D rendering engine: Unity, Unreal Engine, etc.

[0047] Force feedback algorithm: PID controller, etc.

[0048] Operation handle: VR controller or special simulation operation handle that supports force feedback.

[0049] Through the settings of the controller, the present embodiment can complete the calculation and rendering of the user's operation view and operation object in the 3D simulated bronchoscopy scene, and feedback the operation torque through the simulation operation handle, forming a realistic operation experience.

[0050] Preferably, it also includes: A first sensory feedback module is configured to play real device operation sounds at the VR headset according to the user's action data.

[0051] Preferably, further comprising: A second sensory feedback module for spraying liquid simulating lubricant at the VR headset according to the action data of the user to simulate the disinfection operation when the bronchoscopy is completed.

[0052] Optionally, in addition to traditional visual and tactile feedback, the present embodiment also integrates sound feedback and (optional) smell feedback modules to provide a more comprehensive simulation experience. For example, during the simulation operation, the user can hear the real device operation sound, and even smell the disinfectant and lubricant to further enhance the sense of immersion.

[0053] Preferably, further comprising: An intelligent analysis module for real-time monitoring of the action data and comparing the monitoring results with the preset correct operation data, and if the comparison result is incorrect, providing visual guidance feedback information in the 3D simulated bronchoscopy scene.

[0054] Further, through the integrated intelligent analysis module, the present embodiment can monitor the user's operation in real time, and as soon as it finds that the operation is wrong or improper, it immediately provides visual or auditory guidance feedback to help the user correct the error and learn the correct operation method.

[0055] Preferably, further comprising: A storage module for storing the 3D simulated bronchoscopy scene and user data.

[0056] Preferably, further comprising: A network module for downloading new 3D simulated bronchoscopy scenes and updating software from the cloud.

[0057] Preferably, further comprising: A tutorial module for inputting the preset 3D simulated bronchoscopy scene into the VR headset for visual display.

[0058] Specifically, the implementation process of the present embodiment is as follows: 1. Integration of VR headset and simulation operation handle The VR headset is designed to adapt to users with different head shapes, ensuring comfortable wearing and clear vision. The design of the simulation operation handle should as much as possible imitate the real bronchoscope in terms of feel and weight, ensuring the authenticity of the operation feel.

[0059] 2. Cooperative work of position sensor and computer software The position sensor needs to accurately capture every tiny movement of the user and transmit data to the computer in real time. The simulation training software in the computer quickly calculates the impact of the user's operation based on the received data and renders it to the virtual scene in real time, ensuring the continuity and real-time nature of the operation.

[0060] 3. Development and application of adaptive training algorithm An intelligent algorithm is developed to analyze the user's operation proficiency and adjust the difficulty and complexity of the virtual training scene accordingly. The algorithm can also provide personalized learning suggestions and guidance based on the user's learning progress.

[0061] 4. Implementation of multi-sensory feedback system Integrate a high-quality audio feedback system to provide realistic sounds related to the operation.

[0062] (Optional)Integrate a scent generator to release corresponding scents such as disinfectant smell, medicine smell, etc. according to different training scenarios.

[0063] 5. Integration of real-time error correction and guidance system Develop an intelligent analysis module that can analyze user operation data in real time and identify errors or deficiencies in the operation. Once an error is detected, immediate feedback and guidance are provided through the VR headset or audio system to help users learn the correct operation method.

[0064] 6. Optimization of storage module and network module The storage module needs to have sufficient capacity to store a large amount of virtual scene and user data. The network module needs to support high-speed data transmission to ensure the timeliness of virtual scene and software updates.

[0065] Corresponding to the above system, the embodiment also provides a bronchoscope simulation training method based on VR technology, comprising: Generating a 3D simulated bronchoscopy scene; Simulating the operation of a bronchoscope; Capturing action data when the user operates the simulation handle; According to the received action data, calculate and render the user's operation perspective and operation object in the 3D simulated bronchoscopy scene, and feedback the operation torque through the simulation handle to form various operation experiences; According to the action data, evaluate the proficiency to obtain the operation proficiency, and dynamically adjust the difficulty of the 3D simulated bronchoscopy scene according to the operation proficiency.

[0066] The beneficial effects of the present application are as follows: (1) The bronchoscope simulation training device can provide a highly realistic, interactive, easy-to-learn and operate, and can effectively improve the operation skills and clinical response ability of doctors and medical students.

[0067] (2) The present application combines virtual reality technology and bronchoscope operation characteristics to provide a more realistic simulation training environment. Through the cooperation of the VR helmet and the simulation operation handle, the user can experience the real operation experience, thereby improving the operation skill. At the same time, the design of the tutorial module can help novice doctors to better learn and master the bronchoscope examination technology. In addition, the storage module and the network module of the present application enable users to conduct simulation training anytime and anywhere, and download new virtual scenes and update software to meet the needs of continuous learning.

[0068] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the method disclosed by the embodiments, since it corresponds to the system disclosed by the embodiments, the description is relatively simple, and the relevant part can be referred to the system part.

[0069] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above embodiment description is only used to help understand the method and core idea of the present application. For those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A bronchoscopy simulation training system based on VR technology, characterized in that, include: VR headsets are used to generate 3D simulations of bronchoscopy examination scenarios; Simulated operating handle, used to simulate the operation of a bronchoscope; Position sensors are used to capture motion data when a user operates a simulated control handle; The controller is used to calculate and render the user's operating perspective and operating object in the 3D simulated bronchoscopy examination scene based on the received motion data, and to provide feedback on the operating torque through the simulated operating handle to form various operating experiences. An adaptive training algorithm chip is used to evaluate proficiency based on the action data, obtain operational proficiency, and dynamically adjust the difficulty of the 3D simulated bronchoscopy examination scenario based on the operational proficiency.

2. The bronchoscopic simulation training system based on VR technology according to claim 1, characterized in that, Also includes: The first sensory feedback module is used to play realistic device operation sounds on the VR headset based on the user's action data.

3. The bronchoscopic simulation training system based on VR technology according to claim 1, characterized in that, Also includes: The second sensory feedback module is used to spray simulated lubricant liquid onto the VR headset based on the user's motion data to simulate the disinfection procedure after a bronchoscopy.

4. The bronchoscopic simulation training system based on VR technology according to claim 1, characterized in that, Also includes: The intelligent analysis module is used to monitor the action data in real time and compare the monitoring results with preset correct operation data. If the comparison result is incorrect, visual guidance feedback information is provided to the 3D simulated bronchoscopy examination scenario.

5. The bronchoscopic simulation training system based on VR technology according to claim 1, characterized in that, Also includes: The storage module is used to store the 3D simulated bronchoscopy examination scene and user data.

6. The bronchoscopic simulation training system based on VR technology according to claim 1, characterized in that, Also includes: Network module: Used to download new 3D simulated bronchoscopy examination scenarios and update software from the cloud.

7. The bronchoscopic simulation training system based on VR technology according to claim 1, characterized in that, Also includes: The tutorial module is used to input preset 3D simulated bronchoscopy examination scenarios into the VR headset for visualization display; Network module: Used to download new 3D simulated bronchoscopy examination scenarios and update software from the cloud.

8. A bronchoscopic simulation training method based on VR technology, characterized in that, include: Generates a 3D simulation of a bronchoscopy examination scenario; Simulates the operation of a bronchoscope; Capture motion data when the user operates the simulated control handle; The user's operating perspective and operating objects in the 3D simulated bronchoscopy examination scene are calculated and rendered based on the received motion data, and the operating torque is fed back through the simulated operating handle to form various operating experiences. Based on the action data, a proficiency assessment is performed to obtain the operational proficiency level, and the difficulty of the 3D simulated bronchoscopy examination scenario is dynamically adjusted according to the operational proficiency level.

Images