Brain CT medical report generation method based on hierarchical self-attention sequence coding

Abstract

The invention discloses a medical report generation method based on hierarchical self-attention sequence coding. The method comprises the following steps: (1) acquiring a brain CT image and corresponding medical report data, and preprocessing the brain CT image and the corresponding medical report data; (2) constructing a feature extractor; (3) constructing a sequence processor, and obtaining an image feature code VNSA containing information of each adjacent fault block and a three-dimensional brain CT image feature code VSA based on the whole case after the sequence processor passes through the sequence processor; (4) constructing a decoder; and (5) performing model training. The application and development of deep learning in intelligent medical treatment are rapid, and the automatic generation technology of the medical report for the lung is mature, but the research and invention for the automatic generation of the medical report for the brain CT are vacant; therefore, according to the method the model realizes the coding of the three-dimensional brain CT data, and combines the coding with a language model in the field of image description to realize the automatic generation of the medical report of the CT image.

Description

technical field [0001] The invention relates to two fields of computer vision and natural language processing. Aiming at the task of automatically generating reports in the medical field, a method for generating medical reports based on hierarchical self-attention sequence coding is designed. Background technique [0002] 90% of the medical data comes from medical images, and the corresponding medical reports are written by radiologists with professional medical knowledge and experience based on the images, which are the key basis for the attending doctor to diagnose diseases and clarify treatment plans. However, radiologists have always been faced with huge challenges: high knowledge threshold, long training period, heavy tasks, rising rate of missed diagnosis, and medical imaging data is still increasing year by year, the growth rate and work efficiency of radiologists are not enough to cope Such data growth trend. This means that radiologists will be under more and more ...

AI Technical Summary

Problems solved by technology

The uniqueness of brain CT images also brings new challenges to visual semantic matching: the images obtained through cross-sectional scanning have many layers, and the layers are closely related; the lesions are sparse in three-dimensional space, and there is a lot of redundant information.

Images