Surgical navigation interaction system and interaction method based on artificial intelligence and gesture control

Abstract

The invention discloses a surgical navigation interaction system and interaction method based on artificial intelligence and gesture control. The system comprises a camera module, a support module and a calculation module. The method comprises the following steps: 1, acquiring a gesture image through a camera module to form a gesture recognition data set; enabling each gesture to correspond to a gesture code through definition; 2, establishing a neural network model for gesture recognition through the gesture recognition data set; 3, deploying the neural network model in a calculation module; and 4, collecting a current gesture of a doctor through the camera module during an operation, inputting the current gesture into the calculation module, converting the current gesture into a gesture code corresponding to the gesture in the gesture image, and transmitting the gesture code to the operation navigation workstation to convert the gesture code into a corresponding operation code to realize the operation. The method is convenient to operate and high in gesture recognition accuracy; the artificial intelligence calculation speed is high, and intraoperative interaction is free of time delay; a background area can be eliminated, and misoperation caused by the environment is avoided; disinfection operation is not needed, and a non-contact interaction system can be realized.

Description

technical field [0001] The invention relates to the field of surgical navigation of medical instruments, in particular to a surgical navigation interaction system and interaction method based on artificial intelligence and gesture control. Background technique [0002] At present, surgical navigation systems have been widely used in surgical operations. The application of this system can reduce surgical trauma and assist clinicians to accurately find surgical lesions and complete the operation. [0003] The surgical navigation system needs to perform three-dimensional reconstruction of the patient's medical image, and register the patient space before and during the operation, so as to project the real surgical instruments into the image space in real time, and realize the precise positioning of the operation and the improvement of the operation planning. Implement accurately. However, during the execution of the operation, clinical surgeons are required to frequently opera...

AI Technical Summary

Problems solved by technology

[0005] Patent 201710041574.7 discloses a wearable gesture controller. This type of system needs to add sensing electronic components such as IMU to the hand, so it still cannot solve the problem of preoperative high-temperature disinfection of the gesture controller.

[0006] Patent 202011630163.X discloses an intraoperative remote gesture image control system. A gesture recognition system based on artificial intelligence technology is used in the surgical image control system. The system uses an intraoperative optical panoramic lens to recognize gestures. The surgeon performs identity verification, and then tracks and verifies the doctor's gestures in real time; however, due to the large number of people in the operating room, the background is messy, and the operating room operators need to wear protective and sterile equipment such as masks and glasses, identity verification is difficult. Increased probability of gesture recognition errors

[0007] Patent 201710861761.X discloses an intelligent medical system based on gesture recognition control. It does not use traditional image analysis schemes for gesture recognition and manipulation of the intelligent medical system. Due to the lack of artificial intelligence technology, the error rate of gesture recognition is high and the recognition speed is slow. , it is very easy to cause misrecognition of gestures during the operation of medical devices, as well as long time delays, operation freezes, etc.

[0008] Patent 202011189086.9 discloses a method for interactive gesture recognition of doctors and nurses during the operation. This method proposes a deep learning model to solve the speed and accuracy of gesture recognition. However, during the operation, there are many and crowded medical staff around the operating bed. A purely color camera cannot recognize the gesture of a healthcare worker trying to operate the camera from a cluttered image of human hands

Images