A medical image report generation method based on fine-grained attention

By detecting dental lesion regions using DenseNet-121 and YOLOv5, and combining BERT and graph mutual attention mechanisms, dental medical image reports are generated. This solves the problems of fine-grained feature extraction and label scope control, thereby improving the accuracy and orderliness of dental image reports.

CN115861641BActive Publication Date: 2025-12-12ZHEJIANG UNIV OF TECH
0 Cites 0 Cited by

Patent Information

Application Number
CN202211346536.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-31
Publication Date
2025-12-12
Estimated Expiration
2042-10-31

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately extract fine-grained image features and generate ordered medical reports in the dental field, and insufficient control over the scope of labels leads to chaotic diagnostic results in reports.

Method used

The DenseNet-121 convolutional neural network and YOLOv5 object detection tool were used to extract bounding boxes for teeth and lesions. The word embeddings of candidate disease labels were obtained by combining BERT. The fine-grained relationship between the tooth and lesion regions was modeled using geometric relationship graphs and graph mutual attention mechanisms. The report text was generated by the Transformer bridge tower text generator to control the scope of the labels.

Benefits of technology

It improves the accuracy and organization of dental medical imaging reports, generating more accurate and organized text reports.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

A medical image report generation method based on fine-grained attention, comprising: first, extracting teeth and symptom areas in the original X-ray panoramic picture through a pre-trained yolov5 target detection model; extracting features of the original X-ray panoramic picture through a denseNet-121 convolutional neural network, and fusing the features with candidate disease label texts through cross-modal attention to obtain coarse-grained global cross-modal representation. Subsequently, a geometric connection graph is constructed based on the geometric position distribution of the teeth and the symptom areas, so that the teeth and the corresponding lesion areas are aligned in space. Then, the region image features are fused according to the belonging relationship represented by the geometric connection graph through graph attention to obtain fine-grained tooth symptom representation. Finally, the coarse-grained global cross-modal features and the fine-grained tooth symptom representation are input into a transformer bridge tower text generator with two layers of cross-modal attention and controlled by a sentence state forgetting gate to generate a final report text.
Need to check novelty before this filing date? Find Prior Art