Medical image processing method based on hierarchical staggered reasoning and multiple tasks

Abstract

The invention discloses a medical image processing method based on hierarchical staggered reasoning and multiple tasks, and the method comprises the steps: collecting and preprocessing multi-modal clinical data, and constructing an annotation data set based on a report and expert annotation; a multi-modal model is constructed by taking a visual language model as a core, dynamic multi-task curriculum scheduling is realized by a meta scheduler in a training process, supervised fine tuning and instruction type warm-up are carried out in combination with a hierarchical staggered reasoning normal form, and strategy fine tuning is carried out by introducing mixed rewards based on expert preference and a countermeasure criticator. And the reasoning and generating capability of the model is optimized. Reasoning the to-be-diagnosed medical image and the selectable clinical context by using the trained model, and outputting disease classification, focus segmentation, illness severity evaluation, treatment suggestions and structured diagnosis reports, thereby realizing multi-task collaborative medical image intelligent processing. According to the invention, the efficiency, accuracy and clinical practicability of medical image diagnosis can be remarkably improved.

Images