Radiology report generation method fusing disease perception comparison and cross-modal alignment

Through the visual encoder and text encoder combined with disease perception contrast learning and cross-modal memory alignment module, the cross-modal semantic mismatch problem of visual signals and text descriptions in radiological reports is solved, and efficient radiological report generation is achieved, improving the semantic coherence and clinical accuracy of the report.

CN120564947APending Publication Date: 2025-08-29CHONGQING NORMAL UNIVERSITY
View PDF 0 Cites 6 Cited by

Patent Information

Application Number
CN202510648312.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-20
Publication Date
2025-08-29

AI Technical Summary

Technical Problem

Traditionally, manual writing of radiological reports is time-consuming and dependent on the professional level of the physician. There is a cross-modal semantic mismatch between the continuous visual signals of medical images and the discrete text description of diagnostic reports, resulting in deviations in the representation space, and the lack of fine-grained annotations between images and text, making direct feature alignment difficult.

Method used

Features are extracted separately through the visual encoder and the text encoder, and the visual features and text features are mapped to a unified semantic space using the disease perception comparison learning module DACL, and adaptive feature matching is performed through the cross-modal memory alignment module CMA, combining with the three-stage training strategy to optimize the model, including pre-training, reinforcement learning fine-tuning and knowledge distillation.

Benefits of technology

The alignment accuracy of images and text and the recognition rate of key clinical features of report generation are significantly improved, the semantic coherence of the generated report and the accuracy of medical terminology are significantly improved, and the clinical accuracy and generalization ability of the model are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120564947A_ABST
    Figure CN120564947A_ABST
Patent Text Reader

Abstract

The invention relates to the field of radiology reports, and discloses a radiology report generation method fusing disease perception comparison and cross-modal alignment, and the method comprises the following steps: respectively extracting visual features of a medical image and text features of a radiology report through a visual encoder and a text encoder; utilizing a disease perception contrast learning module DACL to map the visual features and the text features to a unified semantic space; self-adaptive feature matching is carried out by using a dynamically updated memory matrix through a cross-modal memory alignment module CMA; a three-stage training strategy optimization model is adopted, and pre-training, reinforcement learning fine adjustment and knowledge distillation are included. Incremental knowledge is dynamically updated through text features, cross-modal feature alignment tasks are focused, coupling with feature representation is avoided, alignment efficiency is improved, a three-stage training strategy is adopted, if student model performance is better, weights are migrated, and clinical accuracy and generalization ability of the model are further enhanced.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Power report generation method, system and equipment based on adaptive learning and medium

    CN120873028A

  • Intelligent report generation method and system based on multi-modal fusion

    CN121052236A

  • A Report Intelligent Generation Method and System Based on Multimodal Fusion

    CN121052236B

  • Signal text data generation and bimodal fusion continuous learning method and system under long-tail distribution

    CN121145936A

  • Internet of Things heterogeneous equipment federal learning method based on cross-modal knowledge distillation

    CN121724107A