Pathological section human-like section reading track generation method based on reinforcement learning

Abstract

The invention provides a pathological section human-like reading track generation method based on reinforcement learning. The method comprises the following steps: constructing a training data set; the training data set comprises a plurality of WSIs and corresponding doctor film reading track data; an RL frame is built, and parameters of the built RL frame are initialized; wSI local image features and a WSI current film reading state are taken as a state S, position movement in eight directions and a preset fixed step length is taken as an action A, and a pathological expectation value output by a PEAN model is taken as a reward R; and training a PEAN model agent based on the deep reinforcement learning Q network and a sequence of the state S, the action A, the reward R and the next state S stored in the experience playback pool to realize iterative optimization of the deep reinforcement learning Q network so as to finally generate a human-like film reading track of which the coincidence degree with the doctor film reading track is greater than or equal to a preset coincidence degree. According to the method, the macroscopic and microcosmic film reading logic of a doctor is reproduced, and the WSI diagnosis efficiency is greatly improved.

Description

Technical Field

[0001] This invention relates to the technical field of pathological slide reading trajectory generation, and more specifically, to a method for generating human-like pathological slide reading trajectories based on reinforcement learning. Background Technology

[0002] Current pathology slide reading trajectory technologies mainly fall into two categories: one is "fixed-path reading," which involves scanning the entire slide image according to a preset grid or region sequence (whole slide imaging, WSI), failing to simulate the dynamic diagnostic logic of physicians; the other is "weakly supervised driving trajectory," which relies on slide-level labels to infer areas of interest, only able to locate lesions but unable to reproduce the clinical reading process of "low-power initial screening → high-power focusing → differential diagnosis." Therefore, the existing pathology slide reading process... Figure 1 The general process is as follows: WSI import → fixed grid scan / weakly supervised region localization → output static lesion region → manual completion of diagnostic logic.

[0003] However, the existing pathological slide reading techniques still have the following defects: (1) Mechanical trajectory: Fixed paths ignore doctors' clinical habits of "prioritizing suspicious areas", resulting in a high proportion of invalid scans (about 30%-50%); (2) Lack of process: Weakly supervised methods only output the result area, which cannot reflect the doctor's thinking process during diagnosis (such as associating "epidermal abnormality" with "dermal infiltration"), making it difficult to use for teaching or reviewing complex cases; (3) Poor compatibility: It cannot work in conjunction with existing pathological AI models, and it is difficult to improve the diagnostic robustness of pathological AI models through trajectory optimization.

[0004] Therefore, we urgently need a novel pathological slide reading method based on reinforcement learning (RL) to solve the technical problems existing in the above-mentioned pathological slide reading methods. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0006] Therefore, in view of the shortcomings of the existing technology, the purpose of this invention is to provide a method for generating human-like reading trajectories of pathological slides based on reinforcement learning.

[0007] To achieve the above objectives, the present invention provides a method for generating human-like pathological slide reading trajectories based on reinforcement learning. This method includes: Step S1: Constructing a training dataset; wherein the training dataset includes: several WSI images and corresponding doctor reading trajectory data; Step S2: Building an RL framework and initializing the parameters of the constructed RL framework; wherein, the local image features of the WSI and the current reading state of the WSI are taken as state S, positional movement in 8 directions and a preset fixed step size is taken as action A, and the pathological expert value output by the PEAN model is taken as reward R; wherein the PEAN model includes PEAN-I; the current reading state of the WSI includes: the magnification data corresponding to the doctor's magnification operation on a specified local area of ​​the WSI, and / or the local area of ​​the already scanned WSI; Step S3: Training PEAN-I of the PEAN model based on the sequence of state S, action A, reward R, and the next state S stored in the experience replay pool, to achieve iterative optimization of the deep reinforcement learning Q network. The network enables PEAN-I to ultimately generate a human-like image reading trajectory with a degree of overlap with the doctor's image reading trajectory greater than or equal to a preset degree; wherein, the human-like image reading trajectory is the corresponding video of the image reading trajectory displayed on WSI.

[0008] Preferably, the method for generating human-like reading trajectories of pathological slides based on reinforcement learning further includes: step S4: outputting the generated human-like reading trajectory to a weakly supervised learning model, so that the weakly supervised learning model can filter the area covered by the doctor's reading trajectory, and train the weakly supervised learning model based on the selected area, so as to improve the performance index ACC of the weakly supervised learning model on the external test set.

[0009] Preferably, the weakly supervised learning model includes: CLAM, and / or ABMIL.

[0010] Preferably, the method for generating human-like pathological slide reading trajectories based on reinforcement learning further includes: step S5: saving the generated human-like reading trajectory and dynamically replaying the human-like reading trajectory to assist in standardized residency training.

[0011] Preferably, the doctor's image viewing trajectory data includes: position information of each viewpoint on the doctor's image viewing viewpoint trajectory, the duration of the doctor's gaze at each viewpoint, and the doctor's WSI operation data; the position information includes: each viewpoint x Coordinates, viewpoints yThe coordinates and timestamps for each viewpoint; the doctor's WSI operation data includes: magnification data corresponding to the doctor's zooming operation on a local specified area of ​​the WSI during the doctor's viewing and annotation of the WSI, and / or translation coordinate data corresponding to the doctor's translation operation on a local specified area of ​​the WSI.

[0012] Preferably, the eight directions are: up, down, left, right, upper left, lower left, upper right, and lower right; the preset fixed step size is 50 μm; the range of the expert value is greater than or equal to 0 and less than or equal to 1; and the value corresponding to the preset overlap is 85%.

[0013] Preferably, the training dataset includes: at least 300 WSI images covering 5 types of skin diseases and corresponding doctor image reading trajectory data; the 5 types of skin diseases are: mole, basal cell carcinoma of the skin, melasma, psoriasis, and melanoma.

[0014] The beneficial effects of this invention are: The method for generating human-like viewing trajectories for pathological slides based on reinforcement learning provided by this invention has the following technical effects: (1) The process is explainable: It can reproduce the doctor's "macroscopic first, microscopic later" logic of reading images. After the trajectory is visualized, it can be used for standardized training teaching, shortening the standardized training cycle of primary care physicians by about 40%; (2) Efficiency optimization: Reduces the invalid scanning area of ​​the generated trajectory by 30%, greatly improving the diagnostic efficiency of WSI; (3) Model empowerment: In collaboration with the weakly supervised model, it solves the problem of "focusing on irrelevant regions", and the AUC of the external test set is improved to 0.984 (4.2% higher than the weakly supervised model alone).

[0015] Additional aspects and advantages of the invention will become apparent from the description which follows, or may be learned by practice of the invention. Attached Figure Description

[0016] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0017] Figure 1 This is a schematic flowchart of a reinforcement learning-based method for generating human-like viewing trajectories for pathological slides, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of eight directions in a reinforcement learning-based method for generating human-like viewing trajectories for pathological slides, according to an embodiment of the present invention. Detailed Implementation

[0018] To better understand the above-mentioned objects, features, and advantages of the present invention, such as Figure 1 and Figure 2 As shown in the accompanying drawings and specific embodiments, the present invention will be further described in detail below. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0020] Figure 1 This is a schematic flowchart of a reinforcement learning-based method for generating human-like viewing trajectories for pathological slides, according to an embodiment of the present invention. Figure 1 As shown, this reinforcement learning-based method for generating human-like viewing trajectories for pathological slides includes: Step S1: Construct the training dataset; the training dataset includes: several WSI images and corresponding doctor's image reading trajectory data; Step S2: Construct the RL framework and initialize the parameters of the constructed RL framework; wherein, the WSI local image features and the current WSI reading state are taken as state S, the position movement in 8 directions and the preset fixed step size are taken as action A, and the pathological expert value output by the PEAN model is taken as reward R; wherein, the PEAN model includes PEAN-I; the current WSI reading state includes: the magnification data corresponding to the doctor's magnification operation on the local specified area of ​​WSI, and / or the local area of ​​WSI that has been scanned; Step S3: Train PEAN-I of the PEAN model based on the sequence of state S, action A, reward R, and next state S stored in the experience replay pool using a deep reinforcement learning Q network to iteratively optimize the deep reinforcement learning Q network so that PEAN-I ultimately generates a human-like image reading trajectory with a degree of overlap with the doctor's image reading trajectory greater than or equal to a preset degree; wherein, the human-like image reading trajectory is the corresponding video of the image reading trajectory displayed on WSI.

[0021] In this embodiment, by replicating the doctor's logic of reading images from macroscopic to microscopic perspectives, the image reading process that ultimately generates a human-like image reading trajectory that highly overlaps with the doctor's image reading trajectory becomes interpretable. At the same time, the generated human-like image reading trajectory reduces invalid scanning areas by 30%, greatly improving the diagnostic efficiency of WSI.

[0022] In one embodiment of the present invention, the method for generating human-like reading trajectories of pathological slides based on reinforcement learning further includes: step S4: outputting the generated human-like reading trajectory to a weakly supervised learning model, so that the weakly supervised learning model can filter the area covered by the doctor's reading trajectory, and train the weakly supervised learning model based on the filtered area, so as to improve the performance index ACC of the weakly supervised learning model on the external test set.

[0023] In this embodiment, by working in conjunction with a weakly supervised model, the problem of its "focus on irrelevant regions" is solved, thereby improving the AUC of the external test set to 0.984 (a 4.2% improvement compared to a weakly supervised model alone).

[0024] In one embodiment of the present invention, the weakly supervised learning model includes: CLAM, and / or ABMIL.

[0025] In one embodiment of the present invention, the method for generating human-like pathological slide reading trajectory based on reinforcement learning further includes: step S5: saving the generated human-like reading trajectory and dynamically replaying the human-like reading trajectory for use in assisting standardized residency training.

[0026] In this embodiment, by saving the generated human-like image reading trajectory and dynamically replaying the human-like image reading trajectory, the trajectory can be visualized for use in standardized training teaching, shortening the standardized training cycle of primary care physicians by about 40%.

[0027] In one embodiment of the present invention, the doctor's image viewing trajectory data includes: position information of each viewpoint on the doctor's image viewing viewpoint trajectory, the duration of the doctor's gaze at each viewpoint, and the doctor's WSI operation data; the position information includes: each viewpoint x Coordinates, viewpoints y The coordinates and timestamps for each viewpoint; the doctor's WSI operation data includes: magnification data corresponding to the doctor's zooming operation on a local specified area of ​​the WSI during the doctor's viewing and annotation of the WSI, and / or translation coordinate data corresponding to the doctor's translation operation on a local specified area of ​​the WSI.

[0028] In one embodiment of the present invention, the eight directions are: up, down, left, right, upper left, lower left, upper right, and lower right; the preset fixed step size is 50 μm; the range of the expert value is greater than or equal to 0 and less than or equal to 1; and the value corresponding to the preset overlap is 85%.

[0029] In one embodiment of the present invention, the training dataset includes: at least 300 WSI images covering 5 types of skin diseases and corresponding doctor image reading trajectory data; the 5 types of skin diseases are: mole, basal cell carcinoma of the skin, melasma, psoriasis, and melanoma.

[0030] The technical solution of the present invention will be illustrated below with a specific embodiment. This specific embodiment of the reinforcement learning-based method for generating human-like viewing trajectories for pathological slides is implemented through the following steps: (1) Step S1: Construct a training dataset; wherein, the training dataset includes: at least 300 WSI images covering 5 types of skin diseases and corresponding doctor reading trajectory data; the 5 types of skin diseases are: mole, basal cell carcinoma of the skin, melasma, psoriasis, and melanoma. The doctor reading trajectory data is the reading trajectory data of at least 5 senior doctors to construct an "expert reading trajectory database".

[0031] Specifically, the doctor's image viewing trajectory data includes: the position information of each viewpoint on the doctor's image viewing viewpoint trajectory, the duration of the doctor's gaze at each viewpoint, and the doctor's WSI operation data; the position information includes: the position information of each viewpoint. x Coordinates, viewpoints y The coordinates and timestamps for each viewpoint; the doctor's WSI operation data includes: magnification data corresponding to the doctor's zooming operation on a local specified area of ​​the WSI during the doctor's viewing and annotation of the WSI, and / or translation coordinate data corresponding to the doctor's translation operation on a local specified area of ​​the WSI.

[0032] (2) Step S2: Build the RL framework and initialize the parameters of the built RL framework; wherein, the WSI local image features and the current WSI image reading state are the state S, the position movement in 8 directions and the preset fixed step size are the actions A, and the pathological expert value output by the PEAN model is the reward R. The initialization of the parameters of the constructed RL framework includes setting the state S to "1920×1080 pixel window image features + current magnification", the action A to 8 directions × 50μm step size, and the reward R to the expert value calculated by the PEAN model (within the range of greater than or equal to 0 and less than or equal to 1).

[0033] like Figure 2 As shown, the eight directions are: up (↑), down (↓), left (←), right (→), upper left (45° above left, i.e., ↖), lower left (45° below left, i.e., ↙), upper right (45° above right, i.e., ↗), and lower right (45° below right, i.e., ↘).

[0034] The PEAN model includes PEAN-C and PEAN-I. The core technical architecture and key technological innovations of the PEAN model are described below.

[0035] I. Core Technical Architecture of the PEAN Model (Two Major Modules Working Together) The PEAN model is divided into PEAN-C (diagnostic knowledge modeling module) and PEAN-I (diagnostic behavior simulation module). The two work together to form a complete technology chain, and their specific architecture and functions are as follows: (1) PEAN-C: This is the diagnostic knowledge modeling module (core functions: lesion diagnosis + expert attention value prediction).

[0036] PEAN-C input: WSI-cut image patches + eye-tracking data (expert gaze points, magnification, and glide path); PEAN-C's core architecture: 1. Feature Extraction Layer: Extracts WSI image patch features based on the CONCH pre-trained encoder, preserving pathological tissue details; 2. Fusion layer: Captures global context (such as the spatial relationship between tumor and normal tissue) through Transformer; 3. Dual-task learning layer (innovation): - Main task: Classification and diagnosis of 5 types of skin lesions (basal cell carcinoma, squamous cell carcinoma, etc.); - Auxiliary task: Predict the "expert attention value" for each image patch (quantify the degree of importance experts attach to the region); - Output: Lesion diagnosis results + lesion area heat map (visualizing the core diagnostic areas of concern to experts).

[0037] (2) PEAN-I: This is the diagnostic behavior simulation module (core function: reproducing the expert reading path).

[0038] - Technical principle: Inverse reinforcement learning is used to "learn" visual decision-making logic from the actual viewing trajectory of experts; - Input: The dynamic operation sequence of a pathologist when reading slides (fixation point movement, magnification switching, field of view sliding); - Output: AI-generated optimized image reading path (simulating the clinical thought chain of experts: "low-magnification initial screening → high-magnification focusing → differential diagnosis"); - Core value: Provides "human-like annotation logic" for subsequent data annotation, solving the "black box" problem of traditional AI annotation.

[0039] II. Key Technological Innovations

[0040] (1) Eye-tracking data fusion technology (data input innovation): - Solving the problem of "how to transform expert implicit visual behavior into computable data": 1. Data Acquisition: Using a high-precision eye tracker at 50 frames per second, the fixation points, dwell times, and visual field movement trajectories of experts during image review are recorded synchronously; 2. Quality Control (Key Patent Process Features): - Pre-calibration: Perform device calibration for 30 seconds before each recording to ensure gaze accuracy; - Abnormal removal: Remove invalid data such as "fixation point outside the background", "eye movement angular velocity > 30° / s (blink / interference)", and "long-term interruption"; 3. Data Compression: Compress 1 hour of image review data to 2-3MB (dedicated EPR format) to meet the storage needs of medical scenarios.

[0041] (2) Dual-task collaborative learning (algorithm innovation): PEAN-C's dual-task approach of "diagnosis + prediction of key information" ensures both "accurate results" and "logical interpretability." - The primary task (diagnosis) is to ensure the accuracy of lesion classification; - The auxiliary task (prediction of attention value) allows AI to "understand" why experts are paying attention to a certain area. Heat maps can be used directly as a basis for annotation, solving the clinical trust problem of traditional AI "only giving results but not reasons".

[0042] (3) Expert knowledge distillation (application innovation): The weakly supervised model is "knowledge distilled" using the expert review path extracted by PEAN-I: - Select high-value image patches (only retaining core areas of interest to experts) for training a weakly supervised model; - Results: The weakly supervised model improved the AUC to 0.936 / 0.912 on the internal / external test sets, which is 12.4% / 9.8% higher than using all image patches, achieving a balance between "weakly supervised efficiency and strong supervised performance".

[0043] In step S2, the PEAN model (Pathology Expertise Acquisition Network) is a deep learning model focusing on pathology data annotation and AI diagnosis. Its core is to capture the implicit experiences of pathology experts during slide reading, such as gaze trajectories and operational sequences, using high-precision eye-tracking technology at 50 frames per second. This is then combined with a dual-task learning algorithm to transform these implicit experiences into structured annotated data and interpretable diagnostic logic. It addresses the pain points of low efficiency (14.2 minutes / slide) and unreliable "black box" AI models in traditional manual annotation, achieving a 25-fold increase in annotation efficiency (36.5 seconds / slide) and an accuracy rate of ≥95%. Simultaneously, it generates expert attention heatmaps, ensuring that the AI ​​diagnostic logic aligns with clinical experts. This makes it suitable for large-scale dataset construction and supports AI medical model training and medical education.

[0044] (3) Step S3: Train PEAN-I of the PEAN model based on the sequence of state S, action A, reward R and next state S stored in the experience replay pool of the deep reinforcement learning Q network, so as to achieve iterative optimization of the deep reinforcement learning Q network, so that the PEAN-I finally generates a human-like reading trajectory with an overlap of greater than or equal to 85% with the doctor's reading trajectory; wherein, the human-like reading trajectory is the corresponding video of the reading trajectory displayed on WSI.

[0045] In step S3, the trajectory generation engine is mainly introduced: based on DRQN ​​(Deep Reinforcement Learning Q-Network), the agent (PEAN-I) is trained to autonomously explore WSI. At each step, it selects an action based on the current state, generating a dynamic trajectory consistent with the doctor's image reading logic (e.g., first locating the lesion range with low magnification, then focusing on cell morphology with high magnification). Specifically, by training the PEAN-I agent, the Q-Network is iteratively optimized based on the "state S - action A - reward R - next state S" sequence stored in the experience replay pool, so that the agent PEAN-I ultimately generates a human-like image reading trajectory with an overlap of greater than or equal to 85% with the doctor's image reading trajectory.

[0046] (4) Step S4: Output the generated human-like image reading trajectory to the weakly supervised learning model so that the weakly supervised learning model can select the area covered by the doctor's image reading trajectory, and train the weakly supervised learning model based on the selected area to improve the performance index ACC of the weakly supervised learning model on the external test set.

[0047] In step S4, the cross-model collaboration interface is mainly introduced: the generated image reading trajectory is output to a weakly supervised learning model (such as CLAM, ABMIL), and the key areas covered by the trajectory are selected for model training to improve diagnostic accuracy.

[0048] (5) Step S5: Save the generated human-like image reading trajectory and dynamically replay the human-like image reading trajectory for use in assisting standardized training. In this step S5, the application of the trajectory is mainly introduced: the generated image reading trajectory is used to assist standardized training (dynamically replaying the human-like image reading process), realizing the visualization of the trajectory for standardized training teaching, which shortens the standardized training cycle of primary care physicians by about 40%.

[0049] In the above specific embodiments, there are alternative solutions for some technical means. 1. Trajectory generation alternative: If hardware computing power is limited, the RL model can be simplified to "rule-based path based on expert trajectory clustering", generating an approximate human-like path by clustering key points of expert trajectories using K-means; 2. Reward mechanism alternative: If the calculation of the expert value is complex, "doctor's stay time + regional diagnostic contribution" can be used as a composite reward to ensure that the trajectory focuses on key areas.

[0050] The key technical points of the present invention are described below in conjunction with the specific embodiment: (1) RL state-action definition for pathological slide reading: For the first time, the “visual scene + operation state” of WSI slide reading is transformed into a state space that can be computed by reinforcement learning, solving the adaptation problem of RL application in pathological scenarios; (2) Reward design based on expertise value: The doctor's professional knowledge is quantified into reward signal to ensure that the generated trajectory conforms to the clinical diagnosis logic (core innovation); (3) Trajectory-model collaboration interface: The slide reading trajectory is seamlessly connected with the weakly supervised model, providing “human-like attention” guidance for the pathological AI model.

[0051] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for generating human-like viewing trajectories for pathological slides based on reinforcement learning, comprising: Step S1: Construct a training dataset; wherein the training dataset includes: several WSI images and corresponding doctor's image reading trajectory data; Step S2: Construct the RL framework and initialize the parameters of the constructed RL framework; wherein, the WSI local image features and the current WSI reading state are taken as state S, the position movement in 8 directions and a preset fixed step size is taken as action A, and the pathological expert value output by the PEAN model is taken as reward R; wherein, the PEAN model includes PEAN-I; the current WSI reading state includes: the magnification data corresponding to the doctor's magnification operation on a local specified area of ​​WSI, and / or the local area of ​​WSI that has been scanned; Step S3: Train PEAN-I of the PEAN model based on the sequence of state S, action A, reward R, and next state S stored in the experience replay pool of the deep reinforcement learning Q network to iteratively optimize the deep reinforcement learning Q network so that PEAN-I finally generates a human-like image reading trajectory with an overlap degree greater than or equal to a preset overlap degree with the doctor's image reading trajectory; wherein, the human-like image reading trajectory is the corresponding video of the image reading trajectory displayed on WSI.

2. The method for generating human-like viewing trajectories for pathological slides based on reinforcement learning according to claim 1, characterized in that, Also includes: Step S4: Output the generated human-like image reading trajectory to the weakly supervised learning model, so that the weakly supervised learning model can select the area covered by the doctor's image reading trajectory, and train the weakly supervised learning model based on the selected area to improve the weakly supervised learning model's performance index ACC on the external test set.

3. The method for generating human-like viewing trajectories for pathological slides based on reinforcement learning according to claim 2, characterized in that, The weakly supervised learning models include: CLAM, and / or ABMIL.

4. The method for generating human-like viewing trajectories for pathological slides based on reinforcement learning according to claim 2, characterized in that, Also includes: Step S5: Save the generated human-like image reading trajectory and dynamically replay the human-like image reading trajectory to assist in standardized training.

5. The method for generating human-like viewing trajectories for pathological slides based on reinforcement learning according to claim 1, characterized in that, The doctor's image reading trajectory data includes: the position information of each viewpoint on the doctor's image reading viewpoint trajectory, the duration of the doctor's gaze at each viewpoint, and the doctor's WSI operation data; the position information includes: each viewpoint x Coordinates, viewpoints y The coordinates and timestamps for each viewpoint; the doctor's WSI operation data includes: magnification data corresponding to the doctor's zooming operation on a local specified area of ​​the WSI during the doctor's viewing and annotation of the WSI, and / or translation coordinate data corresponding to the doctor's translation operation on a local specified area of ​​the WSI.

6. The method for generating human-like viewing trajectories for pathological slides based on reinforcement learning according to claim 1, characterized in that, The eight directions are: up, down, left, right, upper left, lower left, upper right, and lower right; the preset fixed step size is 50μm; the range of the expert value is greater than or equal to 0 and less than or equal to 1; and the value corresponding to the preset overlap is 85%.

7. The method for generating human-like viewing trajectories of pathological slides based on reinforcement learning according to any one of claims 1 to 6, characterized in that, The training dataset includes at least 300 images of skin diseases (WSI) covering 5 types of skin conditions, along with corresponding doctor image reading trajectory data. The 5 types of skin diseases are: mole, basal cell carcinoma, melasma, psoriasis, and melanoma.

Images