Multimodal X-ray image diagnosis report generation method based on reinforcement learning optimization

By optimizing the multimodal neural network through a multi-stage training framework and reinforcement learning mechanism, the adaptability and robustness of multimodal X-ray imaging diagnostic technology under the condition of scarce pediatric X-ray image data are solved, generating high-quality diagnostic reports that are suitable for personalized application scenarios of X-ray imaging diagnosis.

CN121034518APending Publication Date: 2025-11-28SHANGHAI-CHONGQING ARTIFICIAL INTELLIGENCE RES INST
3 Cites 8 Cited by

Patent Information

Application Number
CN202511104354.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-08-07
Publication Date
2025-11-28

AI Technical Summary

Technical Problem

Existing multimodal X-ray imaging diagnostic technologies suffer from insufficient adaptability and robustness when processing data from special groups such as children's X-ray images. In particular, they struggle to generate high-quality diagnostic reports that conform to clinical logic when the amount of data is limited.

Method used

A multi-stage training framework is adopted. First, the basic association between X-ray images and text is established through supervised fine-tuning. Then, a reinforcement learning mechanism is introduced to optimize the model. The policy gradient algorithm of GRPO is used to optimize the multimodal neural network and generate a diagnostic report that conforms to the format and logic of medical reports.

Benefits of technology

Based on limited professional data, it can quickly adapt to specific clinical needs and generate diagnostic reports that are highly professional, accurate, and clinically applicable, thus solving the adaptability and robustness problems of traditional supervised learning methods in complex scenarios.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a multi-modal X-ray image diagnosis report generation method based on reinforcement learning optimization. The method comprises the following steps: acquiring a multi-source public data set and a private data set in a specific range; performing format standardization processing on the X-ray image, performing normalization processing on text information, and constructing a structured public training data pair and a private enhanced data pair; training an end-to-end multi-modal neural network by using the public training data pair in a supervised fine tuning mode; carrying out reinforcement learning on the first-stage end-to-end multi-mode neural network by adopting a strategy gradient algorithm optimization mode of GRPO and using candidate reports and weighted rewards to obtain a second-stage end-to-end multi-mode neural network; and generating a diagnosis report by using the two-stage end-to-end multi-modal neural network. The method is suitable for personalized application scenes of X-ray image diagnosis, can quickly adapt to specific clinical requirements on the basis of limited professional data, and has good practical value and popularization prospect.
Need to check novelty before this filing date? Find Prior Art