Construction method and application of a visual language large model for diagnosis of digestive tract pathology based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's digestive tract pathological diagnosis visual language large model construction method and its application, belong to medical image processing technical field.The method first extracts pathological information by layout analysis and adaptive threshold processing, reorganizes into structured dataset containing inference chain;Second, features and confidence are extracted using visual encoder and multi-branch classifier, and dynamic structured prompt words are generated;Finally, the image and prompt word are input into the multimodal large model, supervised fine-tuning warm start is carried out, and a group of relative strategy optimization algorithm is used to carry out reinforcement learning training with the composite reward function containing format, semantics and diagnostic dimension.The application solves the problem that general model is prone to illusion and lacks reasoning logic in pathological scene, significantly improves the accuracy and logic of pathological report generation.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and medical image processing, and in particular to a method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning, which combines computer vision and natural language processing technologies, and its application. Background Technology

[0002] With the rapid development of digital pathology technology, digital full-field-of-view (WSI) slides have become an important tool for clinical diagnosis, driving a paradigm shift in pathological diagnosis from qualitative observation to quantitative calculation. However, facing the increasing demand for diagnosis and treatment of gastrointestinal diseases, the shortage and uneven distribution of pathologist resources have led to a serious supply-demand imbalance, urgently requiring intelligent auxiliary diagnostic systems.

[0003] In recent years, the emergence of general-purpose visual-language large-scale models (VLLMs, such as Qwen-VL and DeepSeek-VL) has provided a new pathway for end-to-end pathology report generation. Although these general-purpose models perform well on routine tasks, they still face significant challenges in highly specialized pathology diagnosis scenarios:

[0004] First, the visual encoder of general VLLM has difficulty accurately capturing micron-level histological details (such as nuclear atypia, glandular structural disorder, etc.), which leads to the model's biased understanding of images and thus produces incorrect inference results, i.e., the "hallucination" phenomenon.

[0005] Secondly, most existing publicly available pathology datasets only contain "image-text" pairs, and the text often follows rigid clinical templates, lacking a complete logical reasoning chain from morphological feature recognition to diagnostic conclusion generation. This leads to the model being prone to overfitting to the report format during training, while ignoring the inherent logical connections between multimodal information.

[0006] Furthermore, while traditional supervised fine-tuning (SFT) can enable models to learn text format, it often falls short when it comes to domain-specific terminology accuracy and logical coherence in generating long texts. Summary of the Invention

[0007] This invention provides a method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning and its application. It addresses the problems of existing methods for generating pathology reports using general visual language models, which lack precise visual representation of subtle histological features and complete training data for pathological logical reasoning chains. These problems lead to reports that are prone to factual illusions and make it difficult to guarantee the rigor of diagnostic logic and the accuracy of terminology.

[0008] The core technology of this invention is to construct a closed-loop system of "data-model-training". It constructs a structured dataset containing inference chains through adaptive threshold and layout analysis technology, constructs dynamic prompt words using a dual-branch classifier and confidence embedding strategy, and trains a large visual language model by using the Group Relative Policy Optimization (GRPO) reinforcement learning algorithm in conjunction with a composite reward function that includes format, BioBERT semantic similarity and BLEU accuracy.

[0009] In a first aspect, the present invention provides a method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning, the method comprising the following steps:

[0010] Step S1: Construct a structured dataset containing the pathological reasoning chain:

[0011] The original pathology report is obtained, and pathological images, microscopic descriptions and diagnostic conclusions are extracted through layout analysis and adaptive thresholding. The extracted information is then reorganized into structured text data containing reasoning processes.

[0012] Step S2: Construct dynamic prompt words based on classification confidence:

[0013] A visual encoder is used to extract deep visual representations of pathological images, and a multi-branch classifier is used to predict the tissue origin category and confidence level of the pathological images. Based on the relationship between the confidence level and a preset threshold, structured prompt words containing prior information about tissue origin or visual feature guidance information are generated.

[0014] Step S3: Perform multi-stage model training:

[0015] Input pathological images and structured cue words into a multimodal large model;

[0016] First, supervised fine-tuning and warm-starting are performed using a structured dataset to enable the multimodal large model to learn a predefined inference thought chain format; then, a group relative policy optimization algorithm is used for reinforcement learning training.

[0017] In reinforcement learning training, a composite reward function is constructed to optimize the model policy. The composite reward function includes at least the following:

[0018] Format rewards are used to constrain the structural integrity of the output, semantic rewards are used to evaluate the semantic consistency of the microscopic observation description, and diagnostic rewards are used to evaluate the accuracy of the diagnostic conclusions.

[0019] Further, in step S1, pathological images, microscopic descriptions, and diagnostic conclusions are extracted through layout analysis and adaptive thresholding, specifically including:

[0020] The original pathology report is converted into a grayscale image, and an adaptive binarization threshold is calculated based on the statistical features of local neighborhood pixels to perform image binarization processing.

[0021] Morphological operations are used to remove noise interference from images, and table lines are identified and removed by combining edge detection with region analysis.

[0022] Based on layout analysis technology, the spatial layout of text areas, image areas and table areas is identified, and the images of regions of interest, microscopic description text and diagnostic conclusion text are extracted respectively.

[0023] The extracted region of interest images, microscopic description text, and diagnostic conclusion text are reorganized according to the logical order of "thinking-description-response" to form a structured dataset.

[0024] Furthermore, in step S2, the multi-branch classifier includes branch structures corresponding to different organ categories;

[0025] Predicting the tissue origin category and confidence level of pathological images, including:

[0026] The deep visual representation is input into a multi-branch classifier to calculate the probability distribution of the region of interest belonging to each tissue origin;

[0027] Based on the organization with the highest probability of origin, select the corresponding branch structure for further subdivision and classification, and output the highest probability category and its confidence level.

[0028] Further, in step S2, structured prompts containing prior information about the organization's origin or visual feature guidance information are generated. The specific logic is as follows:

[0029] Determine whether the confidence level is higher than the preset confidence threshold;

[0030] If so, the predicted organization origin category will be embedded as a priori information into the structured prompt words;

[0031] If not, then the structured prompts will be embedded with prompts that guide the model to make judgments based on the visual features of the image, instead of directly adopting the predicted organization origin category.

[0032] Furthermore, in step S3, the specific process of the group relative policy optimization algorithm includes:

[0033] For each input query, sample a set of outputs from the old strategy;

[0034] The current policy is optimized by maximizing the objective function, which is calculated based on the probability ratio between the current policy and the old policy, the advantage value, and the KL divergence between the current policy and the reference policy.

[0035] The advantage value is obtained by standardizing the reward value within the sampling group, without the need to build an independent value network.

[0036] Furthermore, the semantic reward is calculated as follows:

[0037] The domain-specific BERT encoder was used to extract feature vectors from the microscopic observation description text generated by the model and the reference description text, respectively.

[0038] Calculate the cosine similarity between two feature vectors and use the cosine similarity as a semantic reward to encourage the generated description to be semantically consistent with the reference description.

[0039] Furthermore, the diagnostic reward is calculated as follows:

[0040] Obtain the diagnostic conclusion text generated by the model and the reference diagnostic text;

[0041] Calculate the BLEU score of the generated diagnostic conclusion text relative to the reference diagnostic text;

[0042] The BLEU score is obtained by calculating the matching accuracy of n-grams of different lengths and then taking a weighted average. The BLEU score is used as a diagnostic reward.

[0043] The calculation rules for the format reward are as follows:

[0044] The detection model outputs text to determine if it contains complete predefined tag pairs and whether the content within the tags is empty and closed.

[0045] If the predefined structure requirements are met, a positive reward value is assigned; if there are missing, duplicate, misaligned, or empty tags, a zero or negative reward value is assigned.

[0046] The composite reward function is a weighted sum of format reward, semantic reward, and diagnostic reward.

[0047] Secondly, this invention provides a large-scale visual language model construction system for gastrointestinal pathology diagnosis based on reinforcement learning, comprising:

[0048] The data construction module is used to perform layout analysis and adaptive threshold processing on the original pathology report, extract key information, and reorganize it into a structured dataset containing reasoning chains.

[0049] The prompt word generation module is used to extract image features and classification confidence using a visual encoder and a multi-branch classifier, and dynamically generate structured prompt words based on the confidence.

[0050] The model training module is used to perform supervised fine-tuning and warm-start of a large multimodal model using a structured dataset, and to perform reinforcement learning training based on a group relative policy optimization algorithm and a composite reward function.

[0051] The composite reward function consists of a weighted average of format reward, semantic reward, and diagnostic reward.

[0052] Thirdly, the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to execute the above-described method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning.

[0053] Fourthly, the present invention provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the above-described method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning.

[0054] The main contributions and innovations of this invention are as follows:

[0055] 1. Significantly reduces hallucinations and improves diagnostic accuracy: By introducing GRPO reinforcement learning and a specially designed composite reward function (especially semantic reward based on BioBERT and diagnostic reward based on BLEU), this invention not only standardizes the report format, but also significantly improves the model's ability to understand and generate pathological terminology, effectively suppressing the generation of hallucinations in medical scenarios by general large models. Experiments show that it significantly outperforms GPT-4o and traditional SFT models in BLEU and BERTScore metrics.

[0056] 2. Enhanced Logical Reasoning Ability: This invention innovatively constructs training data containing a complete reasoning chain of "thinking-description-response," and uses a dual-branch classifier to explicitly embed the classification results and confidence levels of the ROI into the prompt words. This "classify first, then reason" mechanism guides the model to focus on key histological features and automatically triggers fault-tolerant prompts when the confidence level is low, enhancing the logic and robustness of the diagnostic process.

[0057] 3. Improved Training Efficiency and Memory Utilization: The GRPO (Group Relative Policy Optimization) algorithm replaces the traditional PPO algorithm, eliminating the need for a separate value network and significantly reducing memory consumption. This allows for processing larger batch sizes under the same hardware conditions, significantly improving computational efficiency and training stability for multimodal training of high-resolution pathological images.

[0058] 4. High robustness of data processing: The pathology report template recognition algorithm proposed in this invention introduces an adaptive threshold binarization formula and a morphology-based noise processing mechanism, which can effectively cope with original reports with poor scanning quality, uneven lighting, or complex table lines in the clinical environment, thereby constructing a high-quality, high-reliability structured dataset, laying a solid foundation for model training.

[0059] Details of one or more embodiments of the present invention are set forth in the following drawings and description, so that other features, objects and advantages of the invention will be more readily understood. Attached Figure Description

[0060] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0061] Figure 1 This is a schematic diagram of the overall framework for structured extraction of gastrointestinal pathology reports according to an embodiment of the present invention;

[0062] Figure 2 This is a detailed flowchart illustrating the construction of a structured dataset according to an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of a multimodal analysis scheme combining a high-precision visual basic model and domain-specific cues according to an embodiment of the present invention;

[0064] Figure 4 This is a flowchart illustrating the model training phase according to an embodiment of the present invention;

[0065] Figure 5 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0067] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0068] Example 1: A method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning

[0069] This embodiment provides a method for constructing a large-scale visual language model for gastrointestinal pathology diagnosis based on reinforcement learning. This method aims to address the problems of insufficient visual representation capabilities, susceptibility to hallucinations, and lack of logical reasoning chains in general visual language models used in pathology diagnosis. The method mainly includes three core stages: construction of a structured dataset, dynamic cue word generation based on classification confidence, and a two-stage training process based on SFT hot-start and GRPO reinforcement learning.

[0070] Step 1. Construct a structured dataset containing the pathological reasoning chain.

[0071] High-quality data is fundamental to model training. Addressing the issues of inconsistent pathology report formats and the presence of significant noise (such as table lines and artifacts), this embodiment designs a pathology report template recognition and reconstruction algorithm. For example... Figure 1 As shown, the pathology report recognition algorithm parses and standardizes information such as microscopic observation, diagnostic conclusions, and sampling sites in the original pathology report, forming structured text that can be directly used by the model.

[0072] In this embodiment, the specific steps are as follows:

[0073] (1) Image Preprocessing and Adaptive Binarization: First, the original pathology report image is acquired and converted into a grayscale image. To address common issues in scanned documents such as uneven illumination and blurring, this embodiment abandons the fixed global threshold and adopts an adaptive thresholding algorithm for binarization. For each pixel in the image... Its adaptive threshold The size of the area surrounding this pixel is neighborhood window The decision is made, and the calculation formula is as follows:

[0074]

[0075] in, The pixel grayscale value; is the total number of pixels within the neighborhood window; C is a constant (preferably set to 8 in this embodiment) used to fine-tune the adaptive sensitivity; Represents the neighborhood window The internal coordinates are The grayscale value of the pixels (typically ranging from 0 to 255). This step effectively separates text and image content from complex backgrounds.

[0076] (2) Noise removal and table line cleaning: After binarization, morphological operations are used to remove noise and table lines.

[0077] Specifically, pathology report scan images often contain various types of noise, such as salt-and-pepper noise, background noise, or artifacts generated during the scanning process. These noises can interfere with information extraction and recognition. To address this issue, this invention introduces noise removal technology, employing multiple methods to eliminate interfering information in the images.

[0078] Morphological operations: To remove small areas of noise, especially tiny specks or interfering lines, morphological opening and closing operations were used. Opening operations help remove small, isolated areas, while closing operations fill small holes, which is very effective for removing background noise.

[0079] Median filtering: Median filtering smooths images, effectively removing salt-and-pepper noise while preserving edge information. This method is particularly suitable for images containing random noise.

[0080] Furthermore, to address blurry scanned images, this invention employs a deblurring algorithm. This algorithm restores image sharpness through Laplacian sharpening and deconvolution techniques, improving blurry areas in the scanned image, making details more prominent, and thus enhancing the accuracy of subsequent information extraction.

[0081] (3) Page layout analysis and information extraction:

[0082] In pathology reports, tables and frames often hinder information retrieval, especially when table content is mixed with diagnostic conclusions and image areas, affecting the accuracy of recognition. Therefore, removing table lines is a crucial step in pathology report template recognition.

[0083] Table line removal: Fine table lines in the image are removed by applying morphological opening operations (an operation that involves erosion followed by dilation). To ensure that the removal of table lines does not affect other key information in the report, edge detection (such as the Canny algorithm) and region analysis are used to determine the relative position of the table lines to other content in the report, thereby avoiding the accidental deletion of important information.

[0084] Layout Analysis: This invention utilizes layout analysis technology to identify the overall layout structure of a report. Layout analysis helps algorithms identify the relative positions and structural relationships of text, images, tables, and other areas, thus providing a basis for subsequent area segmentation. Specific methods include:

[0085] Text region detection: Determines the precise location of text by analyzing line spacing, character spacing, and the shape of text boxes.

[0086] Image region detection: Based on edge detection and feature matching algorithms, detect the location of pathological images.

[0087] Table area analysis: By analyzing the relative positions of table boundaries, irrelevant grid lines and frames are removed.

[0088] Layout analysis does not rely solely on simple rule matching, but rather learns the spatial structure of documents through deep learning models, enabling it to handle complex report layouts, such as mixed-format reports (containing multiple elements such as tables, images, and text).

[0089] After recognizing the report format, the system divides the image into multiple regions using region segmentation technology: Pathological Image Region (ROI), Microscopic Description Region, Diagnostic Conclusion Region, etc. These regions are then processed and encoded into independent feature vectors for subsequent reasoning and analysis.

[0090] (4) Reasoning Chain Reorganization: This is the key step. The fragmented information extracted above is reorganized according to the logical order of clinical diagnosis to construct training samples that conform to the reasoning chain format. The specific format is as follows:

[0091] <think>(Thinking): The system extracts features such as lesion areas and tumor types from pathological image regions, analyzes the images using a deep learning model, and obtains preliminary pathological analysis results. Specifically, this involves filling in the preliminary reasoning process derived from image feature analysis, such as "mildly disordered glandular structure was observed, and cell nuclei were pen-shaped...".

[0092] (Description): Extract the doctor's diagnostic description from the microscopic description area, combine it with image analysis results for descriptive analysis, and form clear text information. Specifically, this involves filling in the cleaned microscopic observation description text.

[0093] <answer>(Answer / Diagnosis): Combining images and descriptions, generate a final diagnostic conclusion that simulates a doctor's thought process, outputting an accurate pathological diagnosis. Specifically, enter the final pathological diagnosis conclusion.

[0094] This results in structured text data containing the complete reasoning process, providing logical support for subsequent reinforcement learning.

[0095] Finally, the inference chain data generated through the above steps is used as training data input for the reinforcement learning (RL) model. The RL model can gradually master the structured analysis and reasoning process of pathology reports during continuous learning, thereby improving the accuracy and efficiency of diagnosis.

[0096] (5) Construction and annotation of ROI classification dataset: To support the accurate identification of pathological image sources and lesion types by subsequent multi-branch classifiers, this embodiment also specifically constructed an ROI classification dataset with fine histological type annotations. Specifically, regions of interest (ROIs) in gastrointestinal pathological images were accurately identified and independently annotated according to authoritative pathological classification standards. The annotation work was completed independently by committee-certified gastrointestinal pathologists, and a multi-round review mechanism was adopted to ensure accuracy.

[0097] This dataset covers a variety of histological types, and the specific distribution is as follows:

[0098] Gastric data: 11,449 representative gastric regions of interest (ROIs) were screened, covering 11 different histopathological diagnostic categories (categories 0-9). Category 0 includes non-neoplastic lesions such as gastritis and intestinal metaplasia (432 ROIs); categories 1 and 2 correspond to low-grade intraepithelial neoplasia (LIN) and high-grade intraepithelial neoplasia (HIN), respectively; categories 3 to 9 cover various types of gastric cancer, including well / intermediate / poorly differentiated adenocarcinoma, signet ring cell carcinoma, and mucinous carcinoma.

[0099] Intestinal data: 23,677 intestinal regions of interest (ROIs) were screened and divided into four main categories: cancer (5,165 ROIs), high-grade intraepithelial neoplasia (844 ROIs), low-grade intraepithelial neoplasia (6,459 ROIs), and non-tumor tissue (11,209 ROIs). This finely labeled ROI data was used to train a subsequent "multi-branch classifier," enabling it to distinguish between gastrointestinal organ origins and specific lesion subtypes.

[0100] Step 2. Construct dynamic prompt words based on classification confidence.

[0101] To compensate for the shortcomings of general large models in the identification of professional pathological features, this embodiment introduces a "classification first, prompting later" mechanism.

[0102] (1) Visual Feature Extraction and Two-Branch Classification: A UNIv2 model with frozen weights is used as the visual encoder to extract a 1536-dimensional deep visual feature vector from the pathological ROI image. This feature vector is then input into a multi-branch classifier. In this embodiment, the classifier includes independent branch structures for different organ categories (e.g., stomach, intestine). During inference, the probability distribution of the ROI belonging to each tissue origin (e.g., stomach, intestine) is first calculated through a fully connected layer. The branch corresponding to the tissue origin with the highest probability is selected for further subdivision (e.g., if it is determined to be the stomach, the stomach branch is used to determine whether it is gastritis, adenocarcinoma, or a polyp), and the lesion category with the highest probability and its confidence level are output.

[0103] Specifically, the multi-branch classifier is used to determine the tissue origin of the ROI, taking the stomach and intestines as examples:

[0104] The architecture is a dual-branch system, with each branch corresponding to a set of organ categories (stomach or intestines). Each branch consists of two fully connected layers of size D×C and C×D×C, where D=1536 is the input feature dimension and C is the number of organ categories (detailed classification of stomach or intestines).

[0105] The feature vector of each ROI is first mapped to the classification probability space through two fully connected layers, outputting the probability distribution of each category. During inference, for the input ROI, the classifier first calculates the probability distribution of its corresponding organization. Then, based on the organization category with the highest probability for the ROI, the corresponding branch is selected for fast classification, and the highest probability category of the ROI and its confidence level are output.

[0106] To enhance the system's robustness against misclassification or low confidence, this invention adds some logical hints to the Prompt component:

[0107] If a classification error or low confidence (below the threshold of 51%) occurs, the system will not directly use the predicted category as prior information in Prompt. In this case, Prompt will generate a prompt: "The tissue classification result of the current ROI is uncertain. Please make a judgment based on the visual features of the image."

[0108] (2) Dynamic prompt word generation strategy: The system dynamically adjusts the prompt words input to the large model based on the confidence level of the classifier output. The specific logic is as follows: Set a preset confidence threshold (preferably 51% in this embodiment).

[0109] Case A (High Confidence): If the classification confidence is higher than the threshold, the system considers the classification result reliable and embeds the predicted tissue origin and lesion category as prior knowledge into the prompt words. For example: "These images are histopathological images taken from the gastric antrum. According to the gastric classifier, for..." The probability of it not being a tumor is 80%...

[0110] Scenario B (Low Confidence): If the classification confidence is below the threshold, the system considers the classification result questionable. To avoid misleading the large model, the generated prompts do not contain specific classification conclusions, but instead guide the model to make its own judgment. For example: "The current ROI's tissue classification result is uncertain. Please mainly refer to..." Visual features in <think>"Evaluate within the tags..."

[0111] This dynamic strategy effectively enhances the system's fault tolerance and prevents errors from the pre-classifier from being directly transmitted to the large model.

[0112] like Figure 3 As shown, the predicted labels and confidence levels of the ROI are combined with tissue origin information to form structured cues, which are then concatenated with other multimodal information and input into the multimodal model to enhance the model's ability to identify samples from specific tissue origins. For samples from non-gastric or intestinal organs, only tissue origin information is used to generate enhanced cues, allowing the model to maintain its ability to perceive source information, thereby improving the accuracy and robustness of cross-tissue type analysis.

[0113] The complete dynamic text adjustment template is as follows:

[0114] These images are histopathological images taken from (tissue origin). According to the (tissue origin) classifier, for (image), the probabilities of being non-tumor are 80%, 100%, 89%, and 100% respectively, and the probabilities of being tumor are 80%, 100%, 89%, and 100% respectively. Please... <think>< / think> The accuracy of classification is assessed in the middle, the observations under the microscope are described in the middle, and in the middle. <answer>< / answer> The pathological diagnosis is given in the text.

[0115] Step 3. Perform multi-stage model training

[0116] To further enhance the process, this invention designs a two-step strategy to optimize gastrointestinal pathology report generation during the post-reinforcement learning training phase. The training phase process is as follows: Figure 4 .

[0117] (1) The first step is the supervised fine-tuning warm-up phase: Before starting the reinforcement learning phase based on GRPO, the present invention adopts a warm-up strategy through SFT to initialize the multi-modal pathology model. There are mainly three considerations for this design. First, directly exploring the complex task of pathological report generation using a random policy often leads to the "cold start" problem, which is characterized by sparse rewards, unstable value estimation, and non-convergent training. Previous work has shown that combining offline demonstrations or behavior cloning can significantly improve sample efficiency and stabilize the learning dynamics. Second, in the training of large language models, SFT is widely adopted as a standard initialization step before reinforcement learning. Most RLHF (Reinforcement Learning from Human Feedback) paradigms include an SFT phase to inject prior knowledge, improve sample efficiency, and ensure stable policy optimization. Third, to ensure that the subsequent reward signals are interpretable throughout the inference chain, the present invention introduces an explicit chain of thought prompting scheme, where <Thought>...< / Thought> encodes the intermediate reasoning process, <Description>...< / Description> captures the microscopic observations, and <Answer>...< / Answer> gives the final diagnosis output.

[0118] Based on this, the present invention constructs a two-stage SFT data generation process:

[0119] In the first stage, the image, structured prompts, original microscopic findings, and diagnostic conclusions are input into the multi-modal model to generate ROI-level morphological and semantic logical reasoning traces, forming the <Thought> part. For example, a two-stage training strategy of "SFT warm-up + GRPO reinforcement learning" is adopted, and Qwen2.5-VL-7B is used as the base model. The supervised fine-tuning (SFT) warm-up uses the structured dataset constructed in step 1 to perform full-parameter fine-tuning on the model.

[0120] Training parameters: Use cosine annealing learning rate, initial learning rate 1×10 ^-5, batch size is 64, and the number of training epochs is 3.

[0121] Purpose: To enable the model to learn <think> ...... <answer>The output format is modified and basic pathological knowledge is injected to solve the problem of cold start difficulty in reinforcement learning.

[0122] In the second stage, the descriptions and diagnostic conclusions obtained by experts under the microscope are used to fill in the <description> and <response> sections, thereby generating SFT training samples with complete structure and hierarchical annotation, providing high-quality and interpretable supervision signals for the model.

[0123] (2) The second step is the GRPO reinforcement learning post-training stage: Based on the SFT model, the Group Relative Policy Optimization (GRPO) algorithm is used for reinforcement training. By constructing a training set using data completely different from that in the first step, the policy is optimized through the group baseline mechanism, effectively reducing computational overhead. At the same time, the reward is standardized using the advantage function to ensure training stability.

[0124] Algorithm advantages: Compared to the traditional PPO algorithm, GRPO does not require an additional value network to estimate the advantage function. Instead, it calculates the advantage value by sampling a group of outputs for the same problem (group sampling) and utilizing the relative reward of the outputs within the group. This significantly reduces memory consumption and allows for the use of larger batch sizes (Batch Size=32) with limited hardware resources (such as 8×H20 GPUs), thereby improving the stability of multimodal training.

[0125] objective function :

[0126]

[0127]

[0128] in, The parameters represent the current strategy model to be optimized (i.e., the multimodal large model in this invention); This represents the mathematical expectation operation; q represents the input query, which includes the pathological image and the corresponding structured prompt words. G represents the data distribution of the input query; G represents the group size, i.e., the number of output samples for each query q. In this embodiment, G is preferably set to 8;

[0129] This represents the probability distribution of the old policy model (i.e., the model parameters before this update); This represents the probability distribution of the current policy model (i.e., the model parameters being updated); This represents the probability ratio, used to measure the impact of the old and new policies on the same output o. i Changes in generation probability;

[0130] This represents a cutoff function that restricts the probability ratio to a range. Inside; This represents the truncation coefficient, used to prevent excessively large policy updates. In this embodiment, The initial value is set to 0.5, preferably 0.2; β represents the weighting coefficient (regularization coefficient) of the KL divergence term. In this embodiment, the initial value of β is set to 0.5, preferably 0.001; Indicates the current strategy Reference Strategy The Kullback-Leibler divergence between the two is used to constrain model updates from deviating too far from the initial model; This represents the reference model, which is usually the model after training in the SFT stage.

[0131] Indicates the i-th output o i The advantage value is used to evaluate whether the output is better or worse than the average level within the group, by adjusting the reward value within the sampled group. Standardized calculations yielded the following:

[0132]

[0133] in, This represents the original total reward value obtained by the i-th output sample; This represents the set of reward values ​​corresponding to G output samples within the same group; This function represents the average reward value within a group. This represents the standard deviation function used to calculate the within-group reward value.

[0134] (3) Design of composite reward function

[0135] To guide the model in generating high-quality reports, rewards are provided for formatting, microscopic findings, and diagnosis. Furthermore, this invention innovatively incorporates auxiliary prompts into the input, combining ROI classification results, tissue origin, and confidence information. This guides the model to focus on key histological features and diagnostic logic, further reducing illusions and improving the accuracy and consistency between microscopic findings and diagnostic conclusions.

[0136] Specifically, this embodiment designs a composite reward function consisting of three parts. The weighting coefficient λ for each part is set to 1 / 3 to ensure a balanced contribution from each dimension.

[0137]

[0138] Where λ format , λ cap , λ ans is the weight coefficient for each reward item, used to control the importance of the three rewards.

[0139] Format Reward ( ): Ensure that the generated report strictly follows the predefined structure, including <Thought>...< / Thought> (reasoning chain), <Description>...< / Description> (microscopic observation), <Answer>...< / Answer> (final diagnosis). The reward is 1 only when each of the three tags appears once, with correct, non-empty, and enclosed content; if there are missing, repeated, or misaligned tags, the reward is 0.

[0140] Microscopic Observation Reward or Semantic Reward ( ): Evaluate the semantic accuracy of the microscopic description. After the structure verification passes, use the domain-specific BioBERT encoder to extract the feature vectors of the description text (<Description>) generated by the model and the expert reference (reference description) text respectively, and calculate the cosine similarity as a soft reward to encourage the generated description to be semantically consistent with the reference:

[0141]

[0142] where c represents the expert-verified reference microscopic observation description text; represents the microscopic observation description text generated by the model.

[0143] Diagnosis Reward : To ensure that the generated diagnosis text is both accurate and linguistically coherent, the present invention uses the BLEU score as the diagnosis reward function for the <Answer> segment. Calculate the BLEU score of the generated diagnosis text (the diagnosis result generated by the model according to the input ) relative to the reference diagnosis text r. Use the weighted geometric mean:

[0144]

[0145] where n represents the length of the n-gram (n - element grammar), with a value range of 1 to 4; is the precision of the n-gram, optimizing this reward to make the generated diagnosis closer to the real report and more readable; represents the precision of the n-gram match of the generated text relative to the reference text r; = 1 / 4 (the weights of 1-gram to 4-gram are the same), representing the weight coefficient of the nth gram.

[0146] By combining the aforementioned SFT hot-start strategy with GRPO reinforcement training, this invention can significantly improve the accuracy and semantic consistency of microscopic observation and diagnostic conclusions in gastrointestinal pathology reports generated by the model, while ensuring structural integrity. Simultaneously, it reduces the risk of hallucination generation, achieving high-quality automated report generation suitable for clinical use. Furthermore, GRPO eliminates the need for a value network, optimizing directly through policy updates. This avoids the memory consumption associated with multiple networks when handling larger batch sizes under the same hardware conditions, which is particularly important for multimodal training. Multimodal training typically requires processing multiple data sources (such as images and text), which significantly increases the memory burden. GRPO performs exceptionally well in such cases. Especially when processing high-resolution pathological images, GRPO can improve computational efficiency and training speed, which is crucial for tasks requiring substantial computational resources.

[0147] To verify the performance of the model in this invention, qwen2.5v1-7b-instruct was used as the base model. This is a high-performance, lightweight, open-source multimodal language model. During the SFT and GRPO phases, this invention performs full parameter fine-tuning. In the SFT phase, a cosine annealing learning rate plan is used for training with bf16 accuracy, starting from an initial learning rate of 1×10⁻⁶. Starting with 5, cycle 3, warm-up rate 10%, batch size 64, maximum sequence length 1024.

[0148] For the GRPO stage, this invention follows the hyperparameter settings recommended by the original algorithm: KL regularization coefficient β = 0.001, pruning coefficient ε = 0.2, group size G = 8, batch size 32, period 100, and maximum sequence length 1024. This configuration has been proven in previous work to reduce training costs while maintaining stability. Weight coefficient λ format , λ cap and λ ans All are set to 1 / 3 to ensure that the contribution of each reward component is equal. This also prevents the model from over-optimizing a particular reward component, which could lead to neglecting other more important task components.

[0149] All experiments were implemented using the PyTorch framework and ran on an 8×96GB H20 GPU.

[0150] Table 1 below reports a comparison of the various models generating microscopic examination results and diagnostic reports on two digestive pathology datasets. Evaluation baselines included state-of-the-art proprietary models gpt-4o and Gemini-2.5-pro, as well as leading open-source visual language models such as LLaMA-3.2-11B-Vision-Instruct and DeepSeek-VL2. To assess the impact of fine-tuning strategies, this invention further incorporated the base model Qwen2.5-VL-7B, with its supervised fine-tuning variable DiagSFT and reinforcement learning boosting variable DiagR1. All models were evaluated under a uniform cue-based argumentation setting to ensure consistency.

[0151] Table 1

[0152]

[0153] The results clearly demonstrate that DiagR1 outperforms all other models across all tasks. DiagSFT ranks second, while gpt-4o performs best among untrained models. Specifically, in the LZ micro-result generation task, DiagR1 outperforms gpt-4o by 5.90 and 11.83 percentage points in BERTScore and BLEU, respectively. The comparison shows that the introduction of reinforcement learning (RL) significantly improves the BLEU score, validating the model's significant advancement in language generation capabilities, particularly in the expression of pathological terminology. While purely supervised learning methods (such as traditional supervised fine-tuned SFT) can help models learn the basic format and structure of text, they still have limitations in handling domain-specific terminology, concepts, and more precise language expressions. The introduction of reinforcement learning greatly enhances the model's understanding and generation capabilities of pathological terminology.

[0154] Example 2: A Large-Scale Visual Language Model Construction System for Gastrointestinal Pathology Diagnosis Based on Reinforcement Learning

[0155] Based on the same inventive concept, this embodiment provides a system running on an electronic device (such as a computer system containing a high-performance GPU server), the system comprising:

[0156] Data construction module: used to perform step 1 in embodiment 1, including image preprocessing unit (performing adaptive binarization), layout analysis unit and inference chain reorganization unit.

[0157] Prompt word generation module: used to perform step 2 in embodiment 1, integrates UNIv2 visual encoder and dual-branch classifier, and includes a logic judgment unit for selecting prompt word template based on classification confidence.

[0158] Model training module: This module performs step 3 in Example 1 and includes an SFT training unit and a GRPO reinforcement learning unit. It incorporates a composite reward calculation engine for real-time calculation of format, semantic, and diagnostic rewards, which are then fed back to the policy model.

[0159] Example 3

[0160] This embodiment also provides an electronic device, see reference. Figure 5 It includes a memory 404 and a processor 402, wherein the memory 404 stores a computer program and the processor 402 is configured to run the computer program to perform the steps in any of the above method embodiments.

[0161] Specifically, the processor 402 may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0162] Memory 404 may include a mass storage device for data or instructions. For example, and not limitingly, memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 404 may include removable or non-removable (or fixed) media. Where appropriate, memory 404 may be internal or external to a data processing device. In a particular embodiment, memory 404 is non-volatile memory. In a particular embodiment, memory 404 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable read-only memory (PROM), an erasable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), an electrically alterable read-only memory (EAROM), or flash memory, or a combination of two or more of these. Where appropriate, the RAM can be Static Random-Access Memory (SRAM) or Dynamic Random-Access Memory (DRAM). DRAM can be Fast Page Mode Dynamic Random-Access Memory (FPMDRAM), Extended Data Out Dynamic Random-Access Memory (EDODRAM), Synchronous Dynamic Random-Access Memory (SDRAM), etc.

[0163] The memory 404 can be used to store or cache various data files that need to be processed and / or communicated, as well as possible computer program instructions executed by the processor 402.

[0164] The processor 402 reads and executes computer program instructions stored in the memory 404 to implement any of the reinforcement learning-based visual language model construction methods for gastrointestinal pathology diagnosis in the above embodiments.

[0165] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408, wherein the transmission device 406 is connected to the processor 402, and the input / output device 408 is connected to the processor 402.

[0166] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network described above may include wired or wireless networks provided by the communication provider of the electronic device. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 406 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0167] Input / output device 408 is used to input or output information.

[0168] Example 4

[0169] This embodiment also provides a readable storage medium storing a computer program, the computer program including program code for controlling a process to execute the process, the process including the reinforcement learning-based visual language large model construction method for gastrointestinal pathology diagnosis according to Embodiment 1.

[0170] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0171] Generally, various embodiments can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. Some aspects of the invention can be implemented in hardware, while others can be implemented by firmware or software executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that, by way of non-limiting example, these blocks, apparatuses, systems, techniques, or methods described herein can be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0172] Embodiments of the present invention can be implemented by computer software, which may be executable by a data processor of a mobile device, such as a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets, and / or macros can be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. The computer program product may include one or more computer-executable components configured to perform the embodiments when the program is run. The one or more computer-executable components may be at least one piece of software code or a portion thereof. Additionally, it should be noted in this respect that, as Figure 1 Any box in the logical flow can represent a program step, or interconnected logic circuits, boxes and functions, or a combination of program steps and logic circuits, boxes and functions. Software can be stored on physical media such as memory chips or blocks of storage implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as DVDs and their data variants, CDs, etc. The physical medium is a non-transient medium.

[0173] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0174] The above embodiments are merely illustrative of several implementations of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the appended claims.< / answer> < / think> < / think> < / answer> < / think>

Claims

1. A method for constructing a large-scale visual language model for gastrointestinal pathology diagnosis based on reinforcement learning, characterized in that, Includes the following steps: Step S1: Construct a structured dataset containing the pathological reasoning chain: The original pathology report is obtained, and pathological images, microscopic descriptions and diagnostic conclusions are extracted through layout analysis and adaptive thresholding. The extracted information is then reorganized into structured text data containing reasoning processes. The extraction of pathological images, microscopic descriptions, and diagnostic conclusions through layout analysis and adaptive thresholding specifically includes: The original pathology report is converted into a grayscale image, and an adaptive binarization threshold is calculated based on the statistical features of local neighborhood pixels to perform image binarization processing. Morphological operations are used to remove noise interference from images, and table lines are identified and removed by combining edge detection with region analysis. Based on layout analysis technology, the spatial layout of text areas, image areas and table areas is identified, and the images of regions of interest, microscopic description text and diagnostic conclusion text are extracted respectively. The extracted region of interest images, microscopic description text, and diagnostic conclusion text are reorganized according to the logical order of "thinking-description-response" to form the structured dataset; Step S2: Construct dynamic prompt words based on classification confidence: A visual encoder is used to extract deep visual representations of pathological images, and a multi-branch classifier is used to predict the tissue origin category and confidence level of the pathological images. Based on the relationship between the confidence level and a preset threshold, structured prompt words containing prior information about tissue origin or visual feature guidance information are generated. Step S3: Perform multi-stage model training: The pathological images and the structured cue words are input into a multimodal large model; First, supervised fine-tuning and warm-start are performed using the structured dataset to enable the multimodal large model to learn a predefined inference thought chain format; then, a group relative policy optimization algorithm is used for reinforcement learning training. In the reinforcement learning training, a composite reward function is constructed to optimize the model policy, and the composite reward function includes at least: Format rewards are used to constrain the integrity of the output structure, semantic rewards are used to evaluate the semantic consistency of the microscopic observation description, and diagnostic rewards are used to evaluate the accuracy of the diagnostic conclusions. The specific process of the group relative strategy optimization algorithm includes: For each input query, sample a set of outputs from the old strategy; The current policy is optimized by maximizing the objective function, which is calculated based on the probability ratio of the current policy to the old policy, the advantage value, and the KL divergence between the current policy and the reference policy. The advantage value is obtained by standardizing the reward value within the sampling group, without the need to construct an independent value network.

2. The method for constructing a large visual-language model for gastrointestinal pathology diagnosis based on reinforcement learning as described in claim 1, characterized in that, In step S2, the multi-branch classifier includes branch structures corresponding to different organ categories; The predicted tissue origin category and confidence level of the pathological image include: The deep visual representation is input into the multi-branch classifier to calculate the probability distribution of the region of interest belonging to each tissue origin; Based on the organization with the highest probability of origin, select the corresponding branch structure for further subdivision and classification, and output the highest probability category and its confidence level.

3. The method for constructing a large visual-language model for gastrointestinal pathology diagnosis based on reinforcement learning as described in claim 1, characterized in that, In step S2, the specific logic for generating structured prompts containing prior information about the organization's origin or visual feature guidance information is as follows: Determine whether the confidence level is higher than a preset confidence threshold; If so, the predicted organization origin category is embedded as a priori information into the structured prompt word; If not, then the structured prompts will be embedded with prompts that guide the model to make judgments based on the visual features of the image, instead of directly adopting the predicted organization origin category.

4. The method for constructing a large visual-language model for gastrointestinal pathology diagnosis based on reinforcement learning as described in claim 1, characterized in that, The semantic reward is calculated as follows: The domain-specific BERT encoder was used to extract feature vectors from the microscopic observation description text generated by the model and the reference description text, respectively. Calculate the cosine similarity between two feature vectors and use the cosine similarity as the semantic reward to encourage the generated description to be semantically consistent with the reference description.

5. The method for constructing a large visual-language model for gastrointestinal pathology diagnosis based on reinforcement learning as described in claim 1, characterized in that, The diagnostic reward is calculated as follows: Obtain the diagnostic conclusion text generated by the model and the reference diagnostic text; Calculate the BLEU score of the generated diagnostic conclusion text relative to the reference diagnostic text; The BLEU score is obtained by calculating the matching accuracy of n-grams of different lengths and then taking a weighted average. The BLEU score is used as the diagnostic reward. The calculation rules for the aforementioned reward format are as follows: The detection model outputs text to determine if it contains complete predefined tag pairs and whether the content within the tags is empty and closed. If the predefined structure requirements are met, a positive reward value is assigned; if there are missing, duplicate, misaligned, or empty tags, a zero or negative reward value is assigned. The composite reward function is a weighted sum of the format reward, the semantic reward, and the diagnostic reward.

6. A system for implementing the method for constructing a large visual language model for gastrointestinal pathology diagnosis according to any one of claims 1 to 5, characterized in that, include: The data construction module is used to perform layout analysis and adaptive threshold processing on the original pathology report, extract key information, and reorganize it into a structured dataset containing reasoning chains. The prompt word generation module is used to extract image features and classification confidence using a visual encoder and a multi-branch classifier, and dynamically generate structured prompt words based on the confidence. The model training module is used to perform supervised fine-tuning and warm-start of the multimodal large model using the structured dataset, and to perform reinforcement learning training based on the group relative policy optimization algorithm and the composite reward function. The composite reward function is composed of a weighted average of format reward, semantic reward, and diagnostic reward.

7. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to execute the method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning as described in any one of claims 1 to 5.

8. A readable storage medium, characterized in that, The readable storage medium stores a computer program, the computer program including program code for controlling a process to execute the process, the process including the method for constructing a large visual language model for gastrointestinal pathology diagnosis based on reinforcement learning according to any one of claims 1 to 5.

Images