A Fast Segmentation Method for Multimodal Surgical Trajectories Based on Unsupervised Deep Learning

Abstract

The invention discloses a method for rapidly segmenting multimodal surgical trajectories based on unsupervised deep learning, which belongs to the field of robot-assisted minimally invasive surgery. The method first aims at a robot-assisted minimally invasive surgery process. The robot system collects surgical kinematics data and video data; structure for feature extraction on video data. The kinematics data processed by smoothing filter and video data image after feature extraction are sent to the improved TSC model for clustering, and the trajectory pre-segmentation results of n surgical demonstrations are obtained; finally, the PMDD merging algorithm is used to pre-segment each trajectory. The segmentation results are similarly merged, and the merged result is the final trajectory segmentation result. The present invention proposes an optimization scheme and unsupervised deep learning for problems such as over-segmentation, accelerates the extraction speed of video features, improves feature quality, and makes clustering results more accurate.

Description

technical field [0001] The invention belongs to the field of robot-assisted minimally invasive surgery (RMIS), and relates to image feature extraction, deep learning clustering, similarity evaluation, etc., and is specifically a method for fast segmentation of multimodal surgical trajectories based on unsupervised deep learning. Background technique [0002] During robot-assisted minimally invasive surgery (RMIS), the surgical trajectory is recorded by a series of robot kinematics data and video data. By segmenting these surgical trajectories, the surgical process is decomposed into several low-complexity sub-trajectories (sub-actions), which can be used for doctor skill evaluation and demonstration learning. More importantly, by learning these sub-trajectories, the robot can realize the autonomous operation of simple tasks, thereby advancing the automation of robotic surgery. However, due to the complexity of the surgical environment and the differences in the skill levels...

AI Technical Summary

Problems solved by technology

[0007] However, there are still many defects in the existing unsupervised trajectory segmentation methods. First, the slowness of video feature extraction is the main problem affecting medical trajectory segmentation. For example, TSC-VGG, whose video feature extraction time accounts for more than 95% of the total segmentation time , so that the efficiency of unsupervised methods is greatly reduced; secondly, the extracted video features are not significant

The quality of the video features extracted by the existing method is poor, and it may even have a negative effect in the trajectory segmentation, resulting in poor segmentation stability; finally, due to the characteristics of the unsupervised trajectory segmentation method itself, the result will have the problem of over-segmentation, that is, The split segment of the same atomic operation is divided into multiple segments, and some split segment "fragments" appear

Images