Method and system for chest radiograph report generation and lesion localization based on reinforcement learning

Abstract

The application discloses a chest radiograph report generation and lesion positioning method and system based on reinforcement learning, and belongs to the technical field of cross between artificial intelligence and medical image analysis. The method first uses a plurality of source chest radiograph images and query texts to supervise and fine-tune a basic model; then a GRPO reinforcement learning framework is used to optimize a strategy model, and an indicator function guided by token entropy is introduced into a target function, and finally, the optimized strategy model is used to realize end-to-end parallel generation of chest radiograph report texts and lesion boundary box coordinates. The method improves the quality of chest radiograph report generation while realizing the visualization and positioning of lesions, and the integrated output form is more in line with the actual clinical workflow.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and medical image analysis, and in particular relates to a method and system for generating chest X-ray reports and locating lesions based on reinforcement learning. Background Technology

[0002] Chest X-ray (CXR) is the most common and cost-effective medical imaging technique, serving as a first-line tool for diagnosing various chest diseases such as pneumonia, tuberculosis, and lung cancer. In recent years, significant progress has been made in AI-based Vision-Language Models (VLMs) for chest X-rays. Their core objective is to simulate the workflow of radiologists, generating coherent diagnostic reports and accurately locating pathological findings that support the diagnosis.

[0003] Currently, the mainstream approach for developing such multi-task CXR VLMs is large-scale supervised fine-tuning (SFT). This method trains the model on a large dataset with expert annotations, enabling it to initially master report generation and lesion localization. However, this approach has significant limitations: performance improvements are severely restricted by high-quality, large-scale expert-annotated data. Since annotation work requires highly specialized radiologists, obtaining more data to further improve model performance becomes extremely difficult and expensive.

[0004] To overcome the performance bottleneck of SFT, Reinforcement Learning (RL), as an effective post-training technique, has shown great potential in general-domain language and vision-language models. However, its application in multi-task medical VLM is still in its early stages of exploration. A key challenge is that existing RL algorithms (such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO)) are typically designed for a single task objective. When applied to models that need to simultaneously learn multiple different tasks such as report generation and bounding box prediction, optimizing one task often leads to a performance degradation in other tasks, resulting in a performance trade-off.

[0005] For example, in existing technologies, if only the text quality of the generated report is optimized using the standard GRPO framework, it may lead to a decrease in the accuracy of the model in locating lesions; conversely, if only the localization accuracy is optimized, it may impair the clinical accuracy and fluency of the generated report. This negative interference between tasks is a major obstacle faced when applying RL to multi-task medical VLM, because clinical deployment requires the model to maintain a high level of performance across all core capabilities, rather than being biased towards a single aspect. Summary of the Invention

[0006] The technical problem this invention aims to solve is that existing reinforcement learning-based post-training methods for chest X-ray visual-language models suffer from performance imbalance across multiple tasks. Specifically, when using standard single-objective optimization algorithms, optimizing the report generation task may impair the accuracy of lesion localization, or optimizing the localization task may reduce report quality. This results in the model being unable to achieve a synergistic and balanced improvement in diagnostic interpretation and spatial localization capabilities without requiring a large amount of expensive additional labeled data. To address this technical problem, this invention provides a reinforcement learning-based method and system for chest X-ray report generation and lesion localization.

[0007] To achieve the above-mentioned objectives, the present invention specifically adopts the following technical solution:

[0008] In a first aspect, the present invention provides a method for generating chest X-ray reports and locating lesions based on reinforcement learning, comprising the following steps:

[0009] S1. A general vision-language model with region perception capability is used as the base model. The base model is fine-tuned by integrating chest X-ray images from different sources and pre-constructed query text. The labeled data corresponding to the chest X-ray images is enhanced and used as the supervision signal.

[0010] S2. The supervised fine-tuned base model is used as the policy model in the GRPO framework, and reinforcement learning is performed to optimize the policy model. During the optimization process, the policy model is guided to generate chest X-ray report text based on the chest X-ray image through compound prompts. The chest X-ray report text and the bounding box coordinate text output by the policy model itself constitute a complete text sequence for policy model optimization. After calculating the entropy value of each token in the chest X-ray report text, the tokens are classified into two categories according to the entropy value through a preset uncertainty threshold. Different indicator function values ​​are assigned to each type of token to calculate the reinforcement learning objective function. The policy model parameters are updated based on maximizing the reinforcement learning objective function until the policy model converges or reaches the preset upper limit of the number of optimizations.

[0011] S3. Use the optimized strategy model as the final model, input the chest X-ray image to be processed into the final model, and output the chest X-ray report text and the bounding box coordinate text of the lesion in the chest X-ray image from end to end in parallel.

[0012] Based on the above scheme, each step can be implemented in the following preferred manner.

[0013] As a preferred option for the first aspect mentioned above, in S1, the pre-trained general vision-language model Qwen2.5-VL-3B is used as the base model.

[0014] As a preferred option for the first aspect mentioned above, in S1, the chest X-ray images and their labeled data used in the process of supervised fine-tuning of the base model are derived from five public datasets: MIMIC-CXR, VinDR, MS-CXR, PadChest-GR, and CheXpert. The labeled data includes radiological reports, disease bounding boxes, anatomical structure bounding boxes, and disease classification labels.

[0015] As a preferred embodiment of the first aspect above, in S2, the composite prompt is composed of a location query, bounding box coordinate text, and a report generation query, wherein the bounding box coordinate text is generated by the strategy model based on the input chest X-ray image and the location query.

[0016] As a preferred embodiment of the first aspect mentioned above, in S2, for all chest X-ray images within a training batch, the policy model generates a chest X-ray report text corresponding to each chest X-ray image; for each token generated at each time step in the chest X-ray report text, the entropy value of the token is calculated based on the probability distribution output by the policy model; the entropy values ​​of all tokens within the training batch are sorted in descending order, and the sorted tokens are then... An entropy value is used as an uncertainty threshold. Tokens with entropy values ​​greater than or equal to this threshold are designated as high-entropy tokens, and their indicator function value is set to 1. Tokens with entropy values ​​less than this threshold are designated as low-entropy tokens, and their indicator function value is set to 0. These are the preset hyperparameters.

[0017] Furthermore, in S2, for a training batch For all chest X-ray images within the time frame, the strategy model generates a chest X-ray report text for each image. For each time step in the chest X-ray report text... The generated token has its entropy calculated based on the probability distribution output by the policy model. :

[0018]

[0019]

[0020] in, It refers to the size of the vocabulary list; The representation strategy model in the first The vocabulary list selected at the current time step The probability of each word; Indicates the first The probability distribution of the time-step strategy model across the entire vocabulary; Represents probability distribution The logical value vector before passing through Softmax; This represents the temperature coefficient used to adjust the smoothness of the probability distribution. This indicates the Softmax operation; This represents the index of each word in the vocabulary.

[0021] As a preferred embodiment of the first aspect above, in S2, the reinforcement learning objective function is obtained by taking the expected value of the average per-token loss of a group in the GRPO framework. The average per-token loss is obtained by dividing the overall loss of the group by the cumulative sequence length of the group. The overall loss is the sum of the image-level losses of all chest X-ray images in the group, and the cumulative sequence length is the sum of the lengths of the complete text sequences corresponding to all images in the group. The image-level loss of each chest X-ray image is composed of two parts: the first part is the GRPO loss calculated per token in the bounding box coordinate text corresponding to the chest X-ray image, and the second part is the conditional GRPO loss calculated per token in the other text corresponding to the chest X-ray image. The other text is obtained by removing the bounding box coordinate text from the complete text sequence corresponding to the chest X-ray image. The conditional GRPO loss is obtained by multiplying the GRPO loss by the indicator function value corresponding to the token.

[0022] Furthermore, the reinforcement learning objective function The calculation method is as follows:

[0023]

[0024] in, Indicates the parameters of the strategy model; Indicates taking the expected value; This indicates the number of chest X-ray images contained in a group within the GRPO framework; Indicates the number of the group The complete text sequence corresponding to each image; express Length; Indicates the number of the group The bounding box coordinates of each image; express Length; Indicates the token's location index; Indicates GRPO loss; Indicates the number of the group The image in the first The probability ratio at each token position; Indicates the number of the group The image in the first The estimated dominance function at each token position; Indicates the number of the group The image in the first The probability ratio at each token position; Indicates the number of the group The image in the first The estimated dominance function at each token position; Indicates an indicator function; Indicates training batch The corresponding uncertainty threshold; This represents the entropy value of the current token, which is the [number]th [token]. The chest X-ray report text of the first image A token generated at each time step.

[0025] Secondly, the present invention provides a chest X-ray report generation and lesion localization system based on reinforcement learning, comprising:

[0026] The supervised fine-tuning module is used to use a general vision-language model with region awareness as the base model, and to perform supervised fine-tuning of the base model using chest X-ray images integrated from different sources and pre-constructed query text, and to enhance the labeled data corresponding to the chest X-ray images as the supervision signal.

[0027] The optimization module is used to take the supervised fine-tuned base model as the policy model in the GRPO framework and optimize the policy model through reinforcement learning. During the optimization process, the policy model is guided by compound prompts to generate chest X-ray report text based on the chest X-ray image. The chest X-ray report text and the bounding box coordinate text output by the policy model itself constitute a complete text sequence for policy model optimization. After calculating the entropy value of each token in the chest X-ray report text, the tokens are classified into two categories according to the entropy value through a preset uncertainty threshold. Different indicator function values ​​are assigned to each type of token to calculate the reinforcement learning objective function. The policy model parameters are updated based on maximizing the reinforcement learning objective function until the policy model converges or reaches the preset upper limit of the number of optimizations.

[0028] The results generation module is used to take the optimized strategy model as the final model, input the chest X-ray image to be processed into the final model, and output the chest X-ray report text and the bounding box coordinate text of the lesion in the chest X-ray image from end to end in parallel.

[0029] Thirdly, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, can realize the reinforcement learning-based method for generating chest X-ray reports and locating lesions as described in any of the solutions in the first aspect above.

[0030] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the reinforcement learning-based method for generating chest X-ray reports and locating lesions as described in any of the solutions in the first aspect above.

[0031] Fifthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0032] The memory is used to store computer programs;

[0033] The processor is configured to, when executing the computer program, implement the reinforcement learning-based method for generating chest X-ray reports and locating lesions as described in any of the solutions of the first aspect above.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] This invention achieves multi-task balanced optimization. Through multi-task learning and an entropy-guided token selection mechanism, it synergistically improves the quality of report generation and the accuracy of lesion localization, effectively overcoming the performance imbalance problem caused by single-target reinforcement learning.

[0036] This invention has extremely high data utilization efficiency. As a highly efficient reinforcement learning post-training method, it can make the overall performance of the model surpass the effect of traditional multi-round supervised fine-tuning without relying on a large amount of newly added expert-annotated data.

[0037] This invention ensures the stability and clinical applicability of the training process. By focusing on the optimization strategy of key decision tokens, it effectively improves training stability. Its integrated output of diagnostic reports and bounding boxes is more in line with the actual clinical workflow. Attached Figure Description

[0038] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0039] Figure 2 A flowchart illustrating the method of this invention for generating a complete text sequence;

[0040] Figure 3 This is a system block diagram of the present invention;

[0041] Figure 4 This is a schematic diagram of the composition of a computer electronic device according to an embodiment of the present invention. Detailed Implementation

[0042] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0043] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include at least one of those features.

[0044] like Figure 1 As shown, in a preferred embodiment of the present invention, the above-mentioned reinforcement learning-based chest X-ray report generation and lesion localization method includes the following steps S1 to S3. The specific implementation process of each step will be described in detail below.

[0045] S1. A general vision-language model with region awareness is used as the base model. The base model is fine-tuned by integrating chest X-ray images from different sources and pre-constructed query text. The labeled data corresponding to the chest X-ray images is enhanced and used as the supervision signal.

[0046] It should be noted that in S1 of this invention, the pre-trained general vision-language model Qwen2.5-VL-3B is used as the base model. This model natively supports region-aware understanding, providing the necessary foundation for predicting bounding boxes. The structure of Qwen2.5-VL-3B consists of three core parts: a visual encoder, a language model, and a connector. The visual encoder is typically a Vision Transformer, responsible for segmenting the input chest X-ray image into blocks and encoding the image blocks into a series of image feature vectors. The connector is a perceptron-resampler module, responsible for mapping the image feature vectors output by the visual encoder to a semantic space that the language model can understand, and generating a fixed number of visual tokens, which are input into the language model along with the text tokens generated by the word segmenter within Qwen2.5-VL-3B. The language model is a large language model based on the Transformer architecture, responsible for understanding and generating the chest X-ray report text and bounding box coordinates in the form of a text sequence.

[0047] It should also be noted that while Qwen2.5-VL-3B is preferred as the base model in this invention, it is not the only option. Other general-purpose models with strong visual-language understanding capabilities and region awareness, such as the InternVL and Fuyu series, or architectures composed of dedicated visual encoders (such as Swin Transformer) and large language models (such as LLaMA and ChatGLM) connected by a trainable adapter (such as LLaVA-Adapter), can all be used as alternatives, as long as they can process chest X-ray images and output chest X-ray report text and bounding box coordinate information.

[0048] It should be noted that in S1 of this invention, the supervised fine-tuning of the basic model aims to enable the basic model to initially perform two tasks: chest X-ray report generation and lesion localization. During the supervised fine-tuning process, the chest X-ray images and their labeled data are sourced from five public datasets: MIMIC-CXR, VinDR, MS-CXR, PadChest-GR, and CheXpert. The labeled data includes radiological reports, disease bounding boxes, anatomical structure bounding boxes, and disease classification labels. Furthermore, in this embodiment S1, these labeled data are cleaned and uniformly formatted, and augmented using radiological knowledge and a large-scale VLM (Visual Modeling Library) to generate augmented labeled data. The augmented labeled data is no longer merely bounding box coordinate information, but rather information-rich structured data, providing supervisory signals for subsequent model learning.

[0049] Based on the above labeled data, the supervised fine-tuned base model supports five types of tasks: 1) Disease localization can be achieved using predicted disease bounding boxes; 2) Anatomical structure localization can be achieved using predicted anatomical structure bounding boxes; 3) Disease classification can be achieved using predicted disease classification labels; 4) Phrase localization can be achieved by finding the region in the chest X-ray report text that corresponds to the medically relevant phrase specified by the user; 5) Generate chest X-ray report text.

[0050] S2. The supervised fine-tuned base model is used as the policy model in the GRPO framework, and reinforcement learning is performed to optimize the policy model. During the optimization process, the policy model is guided to generate chest X-ray report text based on the chest X-ray image through compound prompts. The chest X-ray report text and the bounding box coordinate text output by the policy model itself constitute a complete text sequence for policy model optimization. After calculating the entropy value of each token in the chest X-ray report text, the tokens are classified into two categories according to the entropy value through a preset uncertainty threshold. Different indicator function values ​​are assigned to each type of token to calculate the reinforcement learning objective function. The policy model parameters are updated based on maximizing the reinforcement learning objective function until the policy model converges or reaches the preset upper limit of the number of optimizations.

[0051] It should be noted that in S2 of this invention, the GRPO (Group Relative Policy Optimization) framework is an algorithm proposed by DeepSeek. Specifically, it is a reinforcement learning method for improving the reasoning ability of large language models (LLM). It belongs to the prior art, and its core advantages are to simplify the training process, reduce resource consumption, and improve sample efficiency.

[0052] In this embodiment, step S2 is the core innovation of the present invention. Specifically, the multi-task reinforcement learning optimization in step S2 is built upon the aforementioned GRPO framework and introduces two innovative mechanisms: First, by designing a composite prompt composed of a location query, bounding box coordinate text, and report generation query, a multi-task learning paradigm is constructed, forcing the policy model to establish an intrinsic connection between lesion localization and chest X-ray report generation. Second, an entropy-guided token selection mechanism is introduced. By calculating token entropy and setting an uncertainty threshold, when calculating the policy gradient, only the gradient contributions generated by high-entropy tokens are considered, while the gradients generated by low-entropy tokens are masked, thereby balancing the learning signals of different tasks and protecting the core medical descriptive capabilities of the policy model. Based on the above two core improvements, a unified GRPO reinforcement learning objective function is used to fully update the lesion localization task, while selectively updating the chest X-ray report generation task, thereby achieving synergistic and balanced improvement of the two core tasks.

[0053] It should be noted that in S2 of this invention, the composite prompt is composed of a location query, bounding box coordinate text, and a report generation query. The bounding box coordinate text is generated by the strategy model based on the input chest X-ray image and the location query.

[0054] In this embodiment S2, firstly, based on the given chest X-ray image... Construct a location query The query is in text form, such as "identify and locate lesions in a chest X-ray image". The policy model then generates an initial response based on the chest X-ray image and the location query, which is the bounding box coordinate text output by the policy model itself. Subsequently, this embodiment generates a query by locating the query, the bounding box coordinates text, and a report. By splicing the pieces together, a composite prompt can be formed. ,in, This indicates a concatenation operation. The report generation query mentioned above is also in text format, such as "Generate a detailed chest X-ray report based on the chest X-ray image and the lesions seen in the chest X-ray image." Then, the strategy model generates the chest X-ray report text based on the chest X-ray image and the composite prompts. This text, together with the bounding box coordinates output by the policy model itself, constitutes a complete text sequence used for policy model optimization. ,like Figure 2 As shown. This chain structure explicitly requires the policy model to combine localization prediction with coherent report generation, thereby bridging the gap between the two tasks and generating richer, more interconnected learning signals.

[0055] It should be noted that in S2 of this invention, for all chest radiographs within a training batch, the policy model generates a chest radiograph report text corresponding to each chest radiograph image; for each time step in the chest radiograph report text... The generated tokens are used to calculate their entropy value based on the probability distribution output by the policy model. The entropy values ​​of all tokens within the training batch are then sorted in descending order. The sorted tokens are then... An entropy value is used as an uncertainty threshold. Tokens with entropy values ​​greater than or equal to this threshold are designated as high-entropy tokens, and their indicator function value is set to 1. Tokens with entropy values ​​less than this threshold are designated as low-entropy tokens, and their indicator function value is set to 0. These are the preset hyperparameters.

[0056] In this embodiment S2, for a training batch For all chest X-ray images within the time frame, the strategy model generates a chest X-ray report text for each image. For each time step in the chest X-ray report text... The generated token has its entropy calculated based on the probability distribution output by the policy model. :

[0057]

[0058]

[0059] in, It refers to the size of the vocabulary list; The representation strategy model in the first The vocabulary list selected at the current time step The probability of each word; Indicates the first The probability distribution of the time-step strategy model across the entire vocabulary; Represents probability distribution The logical value vector before passing through Softmax; This represents the temperature coefficient used to adjust the smoothness of the probability distribution. This indicates the Softmax operation; This represents the index of each word in the vocabulary.

[0060] Research has found that in language generation, the tokens with the highest entropy values ​​(i.e., the tokens with the most model uncertainty and decision-making bifurcation points, referred to as "Fork Tokens") have the greatest impact on the output path. In medical reports, these high-entropy tokens often appear in the reasoning and descriptive logic sections, rather than the specific medical terms themselves. By reducing updates to low-entropy tokens in the report generation task, this effectively prevents the task from excessively dominating the training process, thus reserving sufficient update space for the localization task. Inspired by the above findings, this embodiment calculates the entropy values ​​of all tokens within the training batch and sorts the calculated entropy values ​​in descending order, using the entropy value at the 20th percentile as the uncertainty threshold for the training batch. This allows for the classification of all entropy values, distinguishing between high-entropy and low-entropy tokens. An indicator function is also defined. If and only if the entropy value of the token is greater than or equal to When the entropy is high, the indicator function outputs 1; otherwise, it outputs 0. This indicator function maps the token's entropy value to a binary state: for a high-entropy token, the function outputs 1; for a low-entropy token, the function outputs 0.

[0061] It should be noted that in S2 of this invention, the above-mentioned reinforcement learning objective function is obtained by taking the expected value of the average per-token loss of a group in the GRPO framework. The average per-token loss is obtained by dividing the overall loss of the group by the cumulative sequence length of the group. The overall loss is the sum of the image-level losses of all chest X-ray images in the group, and the cumulative sequence length is the sum of the lengths of the complete text sequences corresponding to all images in the group. The image-level loss of each chest X-ray image is composed of two parts: the first part is the GRPO loss calculated per token in the bounding box coordinate text corresponding to the chest X-ray image, and the second part is the conditional GRPO loss calculated per token in the other text corresponding to the chest X-ray image. The other text is obtained by removing the bounding box coordinate text from the complete text sequence corresponding to the chest X-ray image. The conditional GRPO loss is obtained by multiplying the GRPO loss by the indicator function value corresponding to the token.

[0062] Furthermore, the above reinforcement learning objective function The calculation method is as follows:

[0063]

[0064] in, Indicates the parameters of the strategy model; Indicates taking the expected value; This indicates the number of chest X-ray images contained in a group within the GRPO framework; Indicates the number of in this group The complete text sequence corresponding to each image; express Length; Indicates the number of in this group The bounding box coordinates of each image; express Length; Indicates the token's location index; Indicates GRPO loss; Indicates the number of in this group The image in the first The probability ratio at each token position; Indicates the number of in this group The image in the first The estimated dominance function at each token position; Indicates the number of in this group The image in the first The probability ratio at each token position; Indicates the number of in this group The image in the first The estimated dominance function at each token position; Indicates an indicator function; Indicates training batch The corresponding uncertainty threshold; This represents the entropy value of the current token, which is the [number]th [token]. The chest X-ray report text of the first image A token generated at each time step.

[0065] It should be noted that the above-mentioned advantage function estimates... The reward, which integrates the performance of chest X-ray report generation and lesion localization, is calculated through intragroup standardization, transforming the absolute reward into a relative advantage signal for guiding the strategy model; the absolute reward is composed of the mAP reward for lesion localization and the chest X-ray report generation reward derived from keyword matching based on a large language model.

[0066] Furthermore, for the sake of simplicity, the reinforcement learning objective function is simplified below, and its form is as follows:

[0067]

[0068]

[0069]

[0070]

[0071]

[0072]

[0073] in, That is, a group The cumulative length of the sequence is formed by adding the lengths of the complete text sequences corresponding to all images in the group; That is, a group The overall loss is formed by summing the image-level losses of all chest radiographs within the group; That is, the first in that group Image-level loss for each chest X-ray image. The first part of this image-level loss. For each token in the bounding box coordinate text, directly calculate its GRPO loss; Part Two For each token in the text following the bounding box coordinates, conditional optimization is performed: the GRPO loss is calculated only if the entropy value of the current token is higher than the uncertainty threshold of this training batch; if the entropy value of the current token is lower than the uncertainty threshold, it is not included in the loss calculation. Through this design, the model focuses on optimizing the generated uncertain content during reinforcement learning, while steadily learning structured coordinate information.

[0074] S3. Use the optimized strategy model as the final model, input the chest X-ray image to be processed into the final model, and output the chest X-ray report text and the bounding box coordinate text of the lesion in the chest X-ray image from end to end in parallel.

[0075] It should be noted that in S3 of this invention, the final trained model is permanently saved. In practical applications, when a new chest X-ray image is input, the final model can output two results end-to-end in parallel: one is a high-quality, clinically compliant structured chest X-ray report text, and the other is the precise bounding box coordinate text of the lesions in the chest X-ray image, thereby providing radiologists with comprehensive auxiliary diagnostic information.

[0076] To better demonstrate the specific implementation and technical effects of the present invention, the reinforcement learning-based chest X-ray report generation and lesion localization method shown in steps S1 to S3 of the above preferred implementation is applied to a specific example.

[0077] Example

[0078] The specific implementation process of the reinforcement learning-based chest X-ray report generation and lesion localization method used in this embodiment is as described above and will not be repeated here. For ease of description, the method of the present invention will be referred to as ChestR1 in this embodiment.

[0079] This embodiment integrates chest X-ray (CXR) images from multiple publicly available sources to construct a comprehensive dataset for supervised fine-tuning. Data sources include MIMIC-CXR, VinDR, MS-CXR, PadChest-GR, and CheXpert. All datasets contain chest X-ray reports or lesion localization annotations. This embodiment cleans the interpretation data according to the official guidelines for each dataset and integrates radiological knowledge with a large-scale visual language model, enhancing the localization annotation information through inference trajectories. After a standard training-test set split to avoid data leakage, a training set of 452,876 samples was finally obtained. During the reinforcement learning training phase, this embodiment uses the VinDR dataset—which not only provides complete lesion localization annotations, but also uses an existing large-scale language model to match corresponding chest X-ray reports. The integrated dataset supports five tasks: disease localization, anatomical structure localization, phrase localization, report generation, and disease classification.

[0080] This embodiment designs a comprehensive evaluation system for different task types: in the localization task, the mean accuracy (mAP) is used as the core indicator; for the report generation task, the semantic similarity is measured by BERT Score, the clinical accuracy of chest image features is evaluated by F1 CheXbert, the content coverage is analyzed by ROUGE-L, and the structured clinical reasoning ability is tested by RadGraph F1 score.

[0081] Tables 1 and 2 present the comprehensive comparison results of the proposed ChestR1 method with several key baselines. To ensure fairness, all post-training methods were based on the same supervised fine-tuning model. This embodiment also introduces an extended supervised baseline (SFT_More) and compares its performance with current leading external visual language models. The data in the tables show that ChestR1 exhibits optimal balance and stability across both report generation and lesion localization objectives. Compared to other methods that often excel in one task but sacrifice performance in another, the method of this invention effectively achieves collaborative optimization across multiple tasks.

[0082] Table 1. Comparison of model performance for localization and classification tasks

[0083] Table 2. Performance Comparison of Models for Report Generation Tasks

[0084]

[0085] Specifically, as shown in Table 1, the DAPO model excels in disease and anatomical structure localization (Dis Grd, Ana Grd) tasks, becoming a strong benchmark in this field. However, this "specialization" in localization tasks comes at the cost of sacrificing the model's report interpretation capabilities—as shown in Table 2, it scores low on all report generation metrics, including BERTScore and F1CheXbert. Conversely, while the GRPO method surpasses the SFT baseline on some report quality metrics, its localization accuracy (Phra Grd) and classification performance (Class) decline. In contrast, ChestR1 achieves the best of both worlds: it is highly competitive in localization performance, nearly matching the top DAPO model, while also achieving the best overall performance in report generation.

[0086] Therefore, ChestR1 achieves robust and comprehensive improvements across all tasks compared to the supervised fine-tuning baseline, further confirming the effectiveness of reinforcement learning as a post-training strategy. Importantly, ChestR1's performance on the localization task has reached or even surpassed that of the SFT_More model, which uses more labeled data, achieved without introducing additional expensive expert annotations. As for the report generation task, given its absolute dominance in the original SFT data, ChestR1's gain on this metric is relatively limited, which is in line with expectations.

[0087] In terms of model size, even though the number of parameters in the ChestR1 model is less than that of the external baseline model with 7 billion parameters, the method of this invention, with only 3 billion parameters, still demonstrates high competitiveness. It not only gains an advantage over CheXagent-3B with the same number of parameters, but also shows a balance that RadVLM-7B lacks in comparison: while RadVLM-7B has strong localization capabilities, its report generation quality is a significant weakness, while the method of this invention maintains a high level in both core tasks.

[0088] To verify the effectiveness of the entropy-based token selection strategy, this embodiment conducted systematic ablation experiments, the results of which are shown in Tables 3 and 4. Specifically, for each task, this embodiment added two sets of control experiments: one set (w / ofil.) did not filter tokens, i.e., all tokens were retained for optimization; the other set (rev.) adopted a reverse token selection mechanism, i.e., only the tokens with the lowest entropy values ​​were updated with gradients. The experimental results show that the token filtering mechanism can effectively guide multi-task gradient updates, focusing on key tokens with cross-task gains. If only a multi-task framework is used without selection, it will significantly slow down the model convergence speed; and the reverse selection strategy (updating only low-entropy tokens) is even less effective than the original GRPO method. This comparison fully demonstrates the superiority of the entropy selection strategy proposed in this invention in achieving multi-task optimization balance, highlighting the key role of targeted token selection in multi-task learning.

[0089] Table 3. Ablation Experiment Results for Localization and Classification Tasks

[0090] Table 4. Report Generation Task Ablation Experiment Results

[0091] It should also be noted that the reinforcement learning-based chest X-ray report generation and lesion localization method in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a reinforcement learning-based chest X-ray report generation and lesion localization system corresponding to the reinforcement learning-based chest X-ray report generation and lesion localization method provided in the above embodiments, such as... Figure 3 As shown, it includes:

[0092] The supervised fine-tuning module is used to use a general vision-language model with region awareness as the base model, and to perform supervised fine-tuning of the base model using chest X-ray images integrated from different sources and pre-constructed query text, and to enhance the labeled data corresponding to the chest X-ray images as the supervision signal.

[0093] The optimization module is used to take the supervised fine-tuned base model as the policy model in the GRPO framework and optimize the policy model through reinforcement learning. During the optimization process, the policy model is guided by compound prompts to generate chest X-ray report text based on the chest X-ray image. The chest X-ray report text and the bounding box coordinate text output by the policy model itself constitute a complete text sequence for policy model optimization. After calculating the entropy value of each token in the chest X-ray report text, the tokens are classified into two categories according to the entropy value through a preset uncertainty threshold. Different indicator function values ​​are assigned to each type of token to calculate the reinforcement learning objective function. The policy model parameters are updated based on maximizing the reinforcement learning objective function until the policy model converges or reaches the preset upper limit of the number of optimizations.

[0094] The results generation module is used to take the optimized strategy model as the final model, input the chest X-ray image to be processed into the final model, and output the chest X-ray report text and the bounding box coordinate text of the lesion in the chest X-ray image from end to end in parallel.

[0095] It is understood that the reinforcement learning-based chest X-ray report generation and lesion localization methods described in S1-S3 above can essentially be implemented by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer program product corresponding to the reinforcement learning-based chest X-ray report generation and lesion localization method provided in the above embodiments. This product includes a computer program / instruction, which, when executed by a processor, can implement the reinforcement learning-based chest X-ray report generation and lesion localization method as described in the above embodiments.

[0096] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the reinforcement learning-based chest X-ray report generation and lesion localization method provided in the above embodiments, such as... Figure 4 As shown, it includes a memory and a processor;

[0097] The memory is used to store computer programs;

[0098] The processor is configured to implement the reinforcement learning-based chest X-ray report generation and lesion localization method in the above embodiments when executing the computer program.

[0099] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0100] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the reinforcement learning-based chest X-ray report generation and lesion localization method provided in the above embodiments. The storage medium stores a computer program, which, when executed by a processor, can realize the reinforcement learning-based chest X-ray report generation and lesion localization method in the above embodiments.

[0101] Specifically, in the computer-readable storage medium of the above three embodiments, the stored computer program is executed by a processor, which can perform the aforementioned steps S1 to S3.

[0102] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0103] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0104] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0105] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for generating chest X-ray reports and lesion localization based on reinforcement learning, characterized in that, Includes the following steps: S1. A general vision-language model with region perception capability is used as the base model. The base model is fine-tuned by integrating chest X-ray images from different sources and pre-constructed query text. The labeled data corresponding to the chest X-ray images is enhanced and used as the supervision signal. S2. The supervised fine-tuned base model is used as the policy model in the GRPO framework, and reinforcement learning is performed to optimize the policy model. During the optimization process, the policy model is guided to generate chest X-ray report text based on the chest X-ray image through compound prompts. The chest X-ray report text and the bounding box coordinate text output by the policy model itself constitute a complete text sequence for policy model optimization. After calculating the entropy value of each token in the chest X-ray report text, the tokens are classified into two categories according to the entropy value through a preset uncertainty threshold. Different indicator function values ​​are assigned to each type of token to calculate the reinforcement learning objective function. The policy model parameters are updated based on maximizing the reinforcement learning objective function until the policy model converges or reaches the preset upper limit of the number of optimizations. S3. Use the optimized strategy model as the final model, input the chest X-ray image to be processed into the final model, and output the chest X-ray report text and the bounding box coordinate text of the lesion in the chest X-ray image from end to end in parallel. In S2, the composite prompt is composed of a location query, bounding box coordinate text, and a report generation query. The bounding box coordinate text is generated by the strategy model based on the input chest X-ray image and the location query. In S2, for all chest X-ray images within a training batch, the policy model generates a chest X-ray report text corresponding to each image. For each token generated at each time step in the chest X-ray report text, the entropy value of the token is calculated based on the probability distribution output by the policy model. The entropy values ​​of all tokens within the training batch are sorted in descending order, and the sorted tokens are then... An entropy value is used as an uncertainty threshold. Tokens with entropy values ​​greater than or equal to this threshold are designated as high-entropy tokens, and their indicator function value is set to 1. Tokens with entropy values ​​less than this threshold are designated as low-entropy tokens, and their indicator function value is set to 0. These are preset hyperparameters; In S2, the reinforcement learning objective function is obtained by taking the expected value of the average per-token loss of a group in the GRPO framework. The average per-token loss is obtained by dividing the overall loss of the group by the cumulative sequence length of the group. The overall loss is the sum of the image-level losses of all chest X-ray images in the group, and the cumulative sequence length is the sum of the lengths of the complete text sequences corresponding to all images in the group. The image-level loss of each chest X-ray image is composed of two parts: the first part is the GRPO loss calculated per token in the bounding box coordinate text corresponding to the chest X-ray image, and the second part is the conditional GRPO loss calculated per token in the other text corresponding to the chest X-ray image. The other text is obtained by removing the bounding box coordinate text from the complete text sequence corresponding to the chest X-ray image. The conditional GRPO loss is obtained by multiplying the GRPO loss by the indicator function value corresponding to the token.

2. The method for generating chest X-ray reports and lesion localization based on reinforcement learning as described in claim 1, characterized in that, In S1, the pre-trained general vision-language model Qwen2.5-VL-3B is used as the base model.

3. The method for generating chest X-ray reports and lesion localization based on reinforcement learning as described in claim 1, characterized in that, In S1, the chest X-ray images and their labeled data used in the supervised fine-tuning of the base model are from five public datasets: MIMIC-CXR, VinDR, MS-CXR, PadChest-GR, and CheXpert. The labeled data includes radiological reports, disease bounding boxes, anatomical structure bounding boxes, and disease classification labels.

4. A chest X-ray report generation and lesion localization system based on reinforcement learning, characterized in that, include: The supervised fine-tuning module is used to use a general vision-language model with region awareness as the base model, and to perform supervised fine-tuning of the base model using chest X-ray images integrated from different sources and pre-constructed query text, and to enhance the labeled data corresponding to the chest X-ray images as the supervision signal. The optimization module is used to take the supervised fine-tuned base model as the policy model in the GRPO framework and optimize the policy model through reinforcement learning. During the optimization process, the policy model is guided by compound prompts to generate chest X-ray report text based on the chest X-ray image. The chest X-ray report text and the bounding box coordinate text output by the policy model itself constitute a complete text sequence for policy model optimization. After calculating the entropy value of each token in the chest X-ray report text, the tokens are classified into two categories according to the entropy value through a preset uncertainty threshold. Different indicator function values ​​are assigned to each type of token to calculate the reinforcement learning objective function. The policy model parameters are updated based on maximizing the reinforcement learning objective function until the policy model converges or reaches the preset upper limit of the number of optimizations. The results generation module is used to take the optimized strategy model as the final model, input the chest X-ray image to be processed into the final model, and output the chest X-ray report text and the bounding box coordinate text of the lesion in the chest X-ray image from end to end in parallel. In the optimization module, the composite prompt is composed of a location query, bounding box coordinate text, and a report generation query. The bounding box coordinate text is generated by the strategy model based on the input chest X-ray image and the location query. In the optimization module, for all chest X-ray images within a training batch, the policy model generates a chest X-ray report text corresponding to each image. For each token generated at each time step in the chest X-ray report text, the entropy value of the token is calculated based on the probability distribution output by the policy model. The entropy values ​​of all tokens within the training batch are sorted in descending order, and the sorted tokens are then... An entropy value is used as an uncertainty threshold. Tokens with entropy values ​​greater than or equal to this threshold are designated as high-entropy tokens, and their indicator function value is set to 1. Tokens with entropy values ​​less than this threshold are designated as low-entropy tokens, and their indicator function value is set to 0. These are preset hyperparameters; In the optimization module, the reinforcement learning objective function is obtained by taking the expected value of the average per-token loss of a group in the GRPO framework. The average per-token loss is obtained by dividing the overall loss of the group by the cumulative sequence length of the group. The overall loss is the sum of the image-level losses of all chest X-ray images in the group, and the cumulative sequence length is the sum of the lengths of the complete text sequences corresponding to all images in the group. The image-level loss of each chest X-ray image is composed of two parts: the first part is the GRPO loss calculated per token in the bounding box coordinate text corresponding to the chest X-ray image, and the second part is the conditional GRPO loss calculated per token in the other text corresponding to the chest X-ray image. The other text is obtained by removing the bounding box coordinate text from the complete text sequence corresponding to the chest X-ray image. The conditional GRPO loss is obtained by multiplying the GRPO loss by the indicator function value corresponding to the token.

5. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it can realize the chest X-ray report generation and lesion localization method based on reinforcement learning as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the reinforcement learning-based method for generating chest X-ray reports and locating lesions as described in any one of claims 1 to 3.

7. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the reinforcement learning-based method for generating chest X-ray reports and locating lesions as described in any one of claims 1 to 3.

Images