Operation tool and operation stage identification method based on multi-task learning

Abstract

The invention discloses a multi-task learning-based surgical tool and surgical stage identification method. The method comprises the following steps of: 1) collecting minimally invasive surgery videos and processing the minimally invasive surgery videos to obtain a picture sequence data set; 2) utilizing a Backbone network sharing middle layer to perform preliminary feature extraction on the operation tool and the operation stage in the picture sequence data set, and taking an obtained initial feature map as input of a subsequent feature enhancement module; 3) performing feature fusion on the initial feature map by using a feature enhancement module; and 4) using the double-headed classifier to obtain recognition results of the surgical tool and the surgical stage, using a Sigmoid activation function to calculate one branch of the double-headed classifier to obtain a prediction result of the surgical tool, and using a SoftMax function to calculate the other branch of the double-headed classifier to obtain a prediction result of the surgical stage. According to the method, the feature information of the surgical tools and the feature information of the surgical stages are shared to achieve complementation, associated information between the surgical tools and the surgical stages is fully captured, meanwhile, the feature information is subjected to multi-scale fusion, and geometric expression of deep semantic features is enhanced.

Description

technical field [0001] The invention relates to the technical field of minimally invasive surgical image processing, in particular to a multi-task learning-based surgical tool and surgical stage recognition method. Background technique [0002] Early on, by fixing sensors on surgical tools or acquiring data from surgical robots, it is possible to identify the type of surgical tool used by the surgeon at the current moment and the stage of the operation being performed. But collecting these signals often requires additional equipment on the surgical tools or the surgeon's hands, which can interfere with the normal operation of the procedure. Another class of approaches uses visual features from video or image sequences for automated recognition. Researchers using manual feature extraction methods are limited to their personal domain knowledge, and it is difficult to generalize and describe complex surgical video changes. The method based on deep learning can automatically c...

AI Technical Summary

Problems solved by technology

Second, rapid movement of the camera or smoke from burning tissue can cause blurry images

Third, the camera may not always focus on the operating area during operation, introducing additional noise during video recording

Images