Endoscopic surgery navigation method and system based on augmented reality and deep learning and readable storage medium

Abstract

The invention discloses an endoscopic surgery navigation method and system based on augmented reality and deep learning and a readable storage medium. The method comprises the following steps: acquiring a surgery site image and acquiring the position of an endoscope in a surgery in real time; performing CT or MRI scanning on the first operation target part of the patient before the operation to obtain medical digital image data of the first operation target part; inputting the medical digital image data into the trained deep convolutional network to obtain an accurate second operation target part; performing three-dimensional reconstruction on the second operation target part to obtain a three-dimensional model; acquiring a real-time intraoperative image through an endoscope and registering the real-time intraoperative image with the three-dimensional model; acquiring the posture and the position of the surgical instrument in the surgical scene coordinate system, and registering the posture and the position with the position relation of the three-dimensional model; and outputting the augmented reality surgical navigation image in real time through the augmented reality display device. According to the method, the augmented reality and deep learning are utilized to realize the augmented reality operation navigation image of the endoscope, so that the operation efficiency and safety can be improved.

Description

technical field [0001] The present invention relates to the field of image processing and medical technology, and more specifically, to an endoscopic surgery navigation method, system and readable storage medium based on augmented reality and deep learning. Background technique [0002] Endoscope is a detection instrument that integrates traditional optics, ergonomics, precision machinery, modern electronics, mathematics and software. It includes image sensors, optical lenses, light source lighting, mechanical devices, etc., and can enter the stomach through the mouth or enter the body through other natural orifices and surgical incisions. In modern medicine, lesions that cannot be seen on X-rays can be seen through the endoscope. Augmented reality technology refers to the physical information (visual information, sound, taste, touch, etc.) that is difficult to experience in the real world within a certain time and space range, and then superimposed on the real world to be ...

AI Technical Summary

Problems solved by technology

At present, although VR technology provides great convenience for surgical operations, medical staff still cannot synchronize the preoperative 3D scanning information wi

Images