Cervical tissue slice lesion diagnosis method and device based on reinforcement learning

Abstract

The invention provides a cervical tissue slice lesion diagnosis method and device based on reinforcement learning, and belongs to the technical field of medical diagnosis. The method comprises the following steps: cutting a whole slice image of a standardized cervical tissue slice to be diagnosed according to a preset cutting parameter to obtain an image block, inputting the image block into a visual encoder for feature reasoning analysis to obtain the feature of the whole slice image, calculating the hidden vector of the feature by using a reinforcement learning model, and calculating the hidden vector of the feature. According to the two-dimensional continuous coordinates corresponding to the hidden vectors, indexing of preset iterations is carried out, to-be-diagnosed image block vectors are obtained, an aggregator is used for attention weight distribution, clustering constraint is carried out according to a distribution result, global representation vectors are obtained, the global representation vectors are input into a classification head of the aggregator, and a lesion diagnosis result is output. While high sensitivity is ensured, classification accuracy and focus positioning credibility are remarkably improved, and reliable lesion level judgment is provided for clinicians.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical diagnosis, and relates to a cervical tissue slice lesion diagnosis method and device based on reinforcement learning. BACKGROUND

[0002] Cervical cancer is one of the most common malignant tumors in women worldwide, seriously threatening public health. According to the data released by the International Agency for Research on Cancer (IARC) of the World Health Organization in 2020, there are about 570,000 new cases of cervical cancer worldwide each year, resulting in about 310,000 deaths, and its incidence and mortality rates rank first among female tumors. The occurrence of cervical cancer is highly related to the persistent infection of high-risk human papillomavirus (HR-HPV), especially HPV16 and HPV18 types, which account for the vast majority of carcinogenic cases. Although developed countries have greatly reduced the burden of cervical cancer through HPV vaccination and routine screening, in resource-limited areas, it is still highly prevalent, seriously affecting the quality of life of women and the burden of social medical costs.

[0003] The current commonly used screening methods include Pap smear examination, HPV DNA detection, colposcopic biopsy, and subsequent paraffin section pathological evaluation. The final diagnosis still depends on the subjective interpretation of professional pathologists on the whole slide images (WSI) of cervical tissue sections. However, cervical lesions have high morphological heterogeneity, and precancerous lesions (such as LSIL and HSIL) and benign hyperplasia are difficult to completely distinguish at the tissue level, and the lesion area is unevenly distributed in the section. These factors lead to significant observer differences even among experienced pathologists. In addition, the traditional pathological process not only has a long cycle, high labor intensity, but also relies heavily on professional human resources, and does not have good scalability, making it difficult to support the needs of large-scale population screening. SUMMARY

[0004] The purpose of the present application is to solve the above-mentioned problems existing in the prior art, and to provide a cervical tissue slice lesion diagnosis method based on reinforcement learning.

[0005] The purpose of the present application can be achieved by the following technical scheme: a cervical tissue slice lesion diagnosis method based on reinforcement learning, comprising:

[0006] cropping the whole slice image of the cervical tissue section to be diagnosed after standardization according to a preset cropping parameter, to obtain image blocks corresponding to different cropping parameters;

[0007] inputting the image blocks into a visual encoder, wherein the visual encoder performs feature reasoning analysis on the image blocks to obtain the features of the whole slice image;

[0008] calculate a hidden vector of the feature using a reinforcement learning model, index a two-dimensional continuous coordinate corresponding to the hidden vector for a preset number of iterations, and obtain a to-be-diagnosed image block vector;

[0009] perform attention weight distribution on the to-be-diagnosed image block vector using an aggregator, and obtain a global feature vector by performing clustering constraint according to the distribution result.

[0010] As an optional embodiment of the present application, the image block is input into a visual encoder, and the visual encoder performs feature inference analysis on the image block to obtain features of the whole slice image, including:

[0011] The image block is input into a visual encoder according to a preset batch size, batch inference is performed in a gradient-free environment, and a global summary vector of the whole slice image and a local average vector of the image block are obtained.

[0012] As an optional embodiment of the present application, a hidden vector of the feature is calculated using a reinforcement learning model, a two-dimensional continuous coordinate corresponding to the hidden vector is indexed for a preset number of iterations, and a to-be-diagnosed image block vector is obtained, including:

[0013] The feature is subjected to mean pooling and normalization, and is jointly input into a gated recurrent unit together with a hidden state of the last iteration to obtain a hidden vector corresponding to the feature;

[0014] The hidden vector is mapped into a two-dimensional continuous coordinate using an Actor network;

[0015] The to-be-diagnosed image block vector is indexed according to a pre-divided grid and the two-dimensional continuous coordinate, and the to-be-diagnosed image block vector is re-indexed according to a preset number of iterations based on the indexing result of each time, to obtain a to-be-diagnosed image block vector corresponding to a fraction of the number of iterations.

[0016] As an optional embodiment of the present application, the to-be-diagnosed image block vector is subjected to attention weight distribution using an aggregator, including:

[0017] From the iteration round corresponding to the to-be-diagnosed image block vector, a candidate image block vector is extracted and merged with the to-be-diagnosed image block vector to obtain a feature package of the iteration round;

[0018] Each vector in the feature package is subjected to dimension compression using a double-layer nonlinear mapping network, and an attention weight is output.

[0019] As an optional embodiment of the present application, a global feature vector is obtained by performing clustering constraint on the distribution result using an aggregator, including:

[0020] According to the preset instance vector quantity, the highest attention weight and the lowest attention weight corresponding to the number of vectors are respectively intercepted from the ranking of the attention weight, and the corresponding number of vectors are respectively taken as potential positive examples and potential negative examples;

[0021] The potential positive examples and the potential negative examples are respectively input into a clustering constraint branch, the distance of intra-class vectors is reduced and the distance of inter-class vectors is increased, the vectors in the feature bag after clustering constraint are weighted and summed according to the attention weight, and a global feature vector is output.

[0022] As an optional embodiment of the application, the global feature vector is input into a classification head of the aggregator, and a lesion diagnosis result is output, including:

[0023] The global feature vector is input into the classification head of the aggregator, and a multi-class diagnosis result and the probability of each class diagnosis result are obtained by analysis;

[0024] The class diagnosis result corresponding to the maximum probability is taken as the lesion diagnosis result and is output

[0025] As an optional embodiment of the application, after outputting the lesion diagnosis result, it further includes:

[0026] The attention weight of each vector in the feature bag is mapped to the coordinate position of each vector in the whole slice image, and an attention heat map is generated according to the trajectory of the index to-be-diagnosed image block vector.

[0027] As an optional embodiment of the application, after obtaining the to-be-diagnosed image block vector corresponding to the number of iterations, it further includes:

[0028] The to-be-diagnosed image block vector is input into a multi-layer perception machine for diagnosis scoring, and the obtained score is used for strategy updating and advantage function calculation.

[0029] The application also provides a cervical tissue slice lesion diagnosis device based on reinforcement learning, including:

[0030] An image block acquisition module is configured to crop a whole slice image of a to-be-diagnosed cervical tissue slice after standardization according to preset cropping parameters, and obtain image blocks corresponding to different cropping parameters;

[0031] An analysis feature module is configured to input the image blocks into a visual encoder, and the visual encoder is configured to perform feature inference analysis on the image blocks, and obtain features of the whole slice image;

[0032] An index module is configured to calculate hidden vectors of the features by using a reinforcement learning model, and perform indexing on two-dimensional continuous coordinates corresponding to the hidden vectors for a preset number of iterations, and obtain to-be-diagnosed image block vectors.

[0033] The diagnostic module is used to assign attention weights to the image block vector to be diagnosed using an aggregator, and to perform clustering constraints based on the assignment results to obtain a global representation vector. The global representation vector is then input into the classification head of the aggregator to output the lesion diagnosis result.

[0034] The present invention also provides an electronic device, comprising:

[0035] processor;

[0036] Memory used to store processor-executable instructions;

[0037] The processor is configured to implement the reinforcement learning-based cervical tissue section lesion diagnosis method described above when executing the executable instructions.

[0038] Compared with existing technologies, this invention significantly improves classification accuracy and lesion localization reliability while maintaining high sensitivity. It has high detection accuracy and robustness for cervical cancer and precancerous lesions, and can provide reliable data support and technical tools for pathological diagnosis, scientific research and teaching. It also provides clinicians with reliable basis for lesion level determination and surgical boundary assessment. Attached Figure Description

[0039] Figure 1 This is a block diagram of a cervical tissue section lesion diagnosis method based on reinforcement learning according to an embodiment of the present invention;

[0040] Figure 2 This is an overall flowchart of the cervical tissue section lesion diagnosis method based on reinforcement learning according to an embodiment of the present invention;

[0041] Figure 3 This is a flowchart of the index loop and aggregation process according to an embodiment of the present invention;

[0042] Figure 4 This is a model and parameter configuration diagram of an embodiment of the present invention;

[0043] Figure 5 This is a comparison chart of key evaluation indicators in embodiments of the present invention;

[0044] Figure 6 This is a cervical tissue section lesion diagnostic device based on reinforcement learning, as described in this embodiment of the invention. Detailed Implementation

[0045] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0046] In recent years, with the development of deep learning technology, computational pathology (CPath) has become an important research direction in assisting the diagnosis of cervical cancer. By inputting WSI (Weighted Injection Syndrome) into models such as convolutional neural networks (CNN) and visual transformers (ViT), automatic tissue identification, lesion detection, and pathological grading can be achieved, reducing the human burden and improving consistency. Related studies have shown that deep learning models have achieved diagnostic results close to or even exceeding those of human experts in multiple diseases such as breast cancer, prostate cancer, and lung cancer.

[0047] However, there are still many technical challenges in directly applying artificial intelligence (AI) methods to cervical cancer diagnosis:

[0048] 1. Complex task and detailed classification: Cervical cancer screening often requires the simultaneous and detailed classification of multiple levels of lesions, such as benign, LSIL, HSIL and invasive cancer, which is far beyond the capabilities of traditional binary classification models.

[0049] 2. Difficulty in constructing training data: Annotating cervical pathology slides is time-consuming and labor-intensive, there is a lack of high-quality, large-scale public training sets, and the lesion types are diverse and the lesion areas are widely distributed, which limits the effectiveness of existing models.

[0050] 3. Poor model robustness and limited generalization ability: Most existing studies use single-center, small sample datasets, which makes the models highly sensitive to staining variations, scanning equipment, regional populations, etc., resulting in poor performance in cross-center and cross-population applications.

[0051] Example 1

[0052] like Figure 1 As shown, this embodiment proposes a reinforcement learning-based method for diagnosing cervical tissue section lesions, addressing the aforementioned technical problems. This method aims to solve the existing pathological diagnosis process for cervical squamous intraepithelial lesions, which suffers from high manual workload, strong subjectivity, high misdiagnosis and missed diagnosis rates, and a lack of end-to-end automation. The method includes:

[0053] S1, crop the full slice image of the standardized cervical tissue slice to be diagnosed according to the preset cropping parameters to obtain image blocks corresponding to different cropping parameters;

[0054] S2, The image block is input into the visual encoder, and the visual encoder performs feature reasoning analysis on the image block to obtain the features of the whole slice image;

[0055] S3, use a reinforcement learning model to calculate the hidden vector of the feature, and index the image patch vector to be diagnosed by a preset number of iterations based on the two-dimensional continuous coordinates corresponding to the hidden vector;

[0056] S4. The aggregator is used to assign attention weights to the image block vector to be diagnosed, and clustering constraints are applied according to the assignment results to obtain a global representation vector. The global representation vector is then input into the classification head of the aggregator to output the lesion diagnosis result.

[0057] like Figure 2 As shown, in step S1, firstly, cervical biopsy and conization specimens are digitized into whole-slice images using a high-throughput pathology slide scanner SQ-600 at a magnification of 20×(0.24μm / px), with the output format being .sdpc. Subsequently, the whole-slice images are standardized, including color standardization and foreground segmentation. Specifically, before digitization into whole-slice images, differences in staining depth and development time between different batches can cause color shifts in the digitized whole-slice images, affecting the generalization performance of the models used in subsequent analyses. This embodiment uses Macenko normalization for pre-batch standardization: standardized image I is obtained through four steps: optical density (OD) transformation, principal component analysis (PCA), concentration matrix estimation, and reconstruction. norm To avoid miscutting blank slides or bubble areas as cervical tissue blocks, foreground segmentation is performed. The modules used for foreground segmentation and the key configuration points of each module are as follows: Network structure, the key configuration points are that the U-Net encoder is replaced with a ResNet-34 backbone, and the decoder maintains skip connections; Output channel 1, activating Sigmoid; The loss function is 0.7×Dice Loss + 0.3×BinaryCross-Entropy; Data augmentation, the key configuration points are random flipping, color perturbation, and elastic deformation, where the color perturbation is mainly ±12% saturation / brightness, and in the elastic deformation α=30, σ=4; Throttling, the key configuration points are a fixed threshold θ=0.5 to generate a binary mask MM. After standardization, a cropping task is performed. The scanning magnification M = 20×, the target magnification f, and the sampling rate r = 4. At 20× magnification, a sliding window with a window size s20 = 256px and a step size t20 = 128px is used; at 5× magnification, a sliding window with s5 = 1024px and a step size t5 = 512px is used. The patch (image patch) is saved when M = 1 at the center of the window. Using different magnifications to extract large-field-of-view patches is to preserve wide-area organizational background and contextual information. For the i-th and j-th windows, their top-left corner coordinates (x...)... i ,y j The formula for obtaining the graph is as follows:

[0058] patch(i,j,m)=I norm [x i :x i +s m ,y j :y j +s m]

[0059] The theoretical maximum window number N is approximately:

[0060] N≈[(Ws m ) / t m ]+1×[(Hs m ) / t m +1

[0061] W and H correspond to the width and height of the patch to be cropped, but only windows that meet the foreground segmentation conditions are saved, so the actual N is smaller. The cropping task can be executed by writing a script. In actual application, the script cut_patches_8.0.py can complete the following: automatically call OldSdpc or OpenSL ide to read the image; allocate GPU / CPU resources in multiple threads; save the effective patch as a JPEG named "x_y.jpg" to the ... / Patch / WSI_ID / directory; and generate thumbnails and tissue masks for subsequent quality control.

[0062] In step S2, based on the whole-slice image obtained in S1, a visual encoder is selected according to the diagnostic accuracy and speed requirements. The whole-slice image is input into the visual encoder in "inference mode," and a complete WSI feature representation is extracted from the visual encoder output. This representation contains both macroscopic and fine-grained information, which can comprehensively reflect the potential pathological features in the slice, laying a high-quality foundation of data for subsequent reinforcement learning slice selection and lesion classification.

[0063] In step S3, the instance selection and context enhancement aggregation tasks are jointly completed based on the Actor-Critic reinforcement learning framework and CLAM-RL: 1) The patch feature state of the current slice is encoded using a GRU (Gated Recurrent Unit) network, projected as a hidden vector of uniform dimension, and the action probability distribution is output by the Actor network. The patch index with the most diagnostic value is sampled, supporting 4 rounds of iterative iteration to obtain patch features, i.e., the image patch vector to be diagnosed. 2) The historical state vector is cached in the memory unit and fused with the current state through the Cross-Attention mechanism to achieve information alignment and expression enhancement. 3) The selected patch feature vector is evaluated by the Multi-Layer Perceptron and then backpropagated with the reinforcement learning strategy. The overall training objective jointly includes cross-entropy and PPO loss terms. 4) All screening and fusion processes are integrated end-to-end into the CLAM-RL aggregator to avoid redundant I / O and staged manual operations.

[0064] In step S4, the patch features aggregated with enhanced context are input into the CLAM model for attention weight allocation and clustering constraints to obtain a global representation vector. Finally, the MLP (Multilayer Perceptron) classification head performs four-class prediction of cervical benign, LSIL, HSIL and squamous cell carcinoma on the global representation vector, and outputs the confidence and uncertainty scores of each class for subsequent collaborative decision-making and dynamic verification mechanisms.

[0065] Preferably, the image patch is input into a visual encoder, which performs feature reasoning analysis on the image patch to obtain the features of the whole slice image, including:

[0066] The image blocks are input into the visual encoder according to the preset batch size, and batch inference is performed in a gradient-free environment to obtain the global generalization vector of the whole slice image and the local average vector of the image blocks.

[0067] This embodiment employs a Vision Transformer visual encoder. Next, the root directory of the whole-slide image data is traversed to determine the unique identifier w for each WholeSlideImage (WSI). i Its corresponding path P i Each P i Each folder contains multiple patch files, all originating from the same WSI and cropped at a 20x magnification. The mapping relationship is stored in memory, and P is selected sequentially. i Proceed to the reasoning process. The mapping relationship is as follows: To avoid the impact of variations in staining depth between different batches on feature extraction by the visual encoder, Macenko color normalization, pre-executed on the patch, can be read into the CPU. The optical density transformation is: OD(x) = -log((x+1) / I0); in most implementations, I0 is set to 255 by default. This is because pixel values ​​in digital pathology images are typically encoded using 8 bits (range 0-255), with 255 representing a pure background without sample occlusion (maximum light intensity), consistent with the physical meaning of "blank background light intensity" in optical density calculation. In pathology images, a completely pure background (pixel value 255) may contain a small number of noise points (such as sensor errors or image processing). Choosing 240 as the threshold can filter out the interference of these extreme values. Principal component analysis is used to obtain the staining vector, and then the reference staining vector matrix S is used... ref Reconstructed normalization results: I norm =I0exp(-S ref C), where C is the concentration matrix. After standardization, the patch is loaded into memory according to the set batch size B and awaits inference.

[0068] After entering the inference phase, pre-processed patches are batched into the Vision Transformer visual encoder in "inference mode" according to a preset batch size. The encoder weights are frozen and it operates in a gradient-free environment to ensure minimal memory usage and maximized inference speed. After each batch of images completes inference, two parts of information are extracted from the encoder output: a CLS marker vector, which is the encoder's global summary of the entire slice image content (i.e., the global summary vector); and an arithmetic mean of the patch vector set, which is the average of the feature vectors of all patches within the same slice, yielding a summary of local information for that slice (i.e., the local average vector). Concatenating the "global summary vector" and the "local average vector" can be understood as first placing the CLS marker vector representing the global summary, followed by the arithmetic mean of the patch vector set representing local information, forming a complete WSI feature representation. This representation possesses both macroscopic and fine-grained information, comprehensively reflecting the potential pathological features in the slice, laying a high-quality foundation for subsequent reinforcement learning slice selection and lesion classification.

[0069] In this embodiment, once the feature representation of a WSI is generated, it is immediately serialized in binary format and written to a pre-specified output directory named "slice number.h5". The output file WSI_ID.h5 for each WSI contains: features: an N×1024 floating-point matrix and coordinates: N×2 integer coordinates;

[0070] If running in multi-GPU parallel mode, each GPU process first writes the generated features to its own independent subdirectory, thus avoiding concurrent write conflicts. After all processes have completed inference, the files in the subdirectories are merged and archived by a unified process. At this point, the feature generation and saving process of the whole slice image is complete, and the output feature file can be directly read by the downstream reinforcement learning slice selection module and multi-instance classifier without additional format conversion, thus ensuring the efficiency and consistency of data flow.

[0071] The visual encoder can be selected based on diagnostic accuracy and speed requirements, such as ViT-B / 16, ViT-L / 14, DINO-v2-Giant, etc. Taking CERV-ViT-Base as an example, its hidden dimension is 768, and it uses an improved SwiGLU activation. Pre-trained weights are read from ckpt_path, and their positions are embedded into pos-embed and automatically interpolated to the current input resolution. If the model is trained on multiple GPUs, the state_dict key names are first adapted, and then model.load_state_dict() is called to complete the loading. The model is then migrated to the CUDA device cuda:i and model.eval() is executed. Gradient calculation is disabled during subsequent inference stages to save GPU memory.

[0072] Preferably, a reinforcement learning model is used to calculate the hidden vector of the feature, and the image patch vector to be diagnosed is obtained by indexing the two-dimensional continuous coordinates corresponding to the hidden vector for a preset number of iterations. This includes:

[0073] The features are averaged and normalized, and then combined with the hidden state of the previous iteration and input into the gated recurrent unit to obtain the hidden vector corresponding to the features.

[0074] The hidden vectors are mapped to two-dimensional continuous coordinates using an Actor network;

[0075] The image block vector to be diagnosed is indexed according to the pre-divided grid and the two-dimensional continuous coordinates. Based on the indexing result of each time, it is indexed again according to the preset number of iterations to obtain the number of image block vectors to be diagnosed corresponding to the number of iterations.

[0076] like Figure 3 As shown, the 20×patch features obtained in step S2 are sampled. Specifically, F∈R N×1024 Perform LayerNorm and Sigmoid transformations to obtain the normalized matrix. In this stage, F represents the previously obtained features: an N×1024 floating-point matrix. Then, mean pooling is performed on the 20×patch features, and the result is normalized based on the matrix. Normalization is performed to obtain the initial state vector:

[0077]

[0078] The initial state vector is input into the GRU to obtain the first-round hidden state h0, and a history memory buffer H is created. In this embodiment, the maximum length of H is L. max =4.

[0079] The algorithm also employs the Actor-Critic structure of SmartPatchRL (Smart Patch Reinforcement Learning framework). SmartPatchRL is a hybrid method combining patch-based local perception and reinforcement learning (RL). When selecting the image patch vector to be diagnosed, the features of the current candidate patch are first averaged and normalized to form an overall view. The result is then input into the GRU along with the hidden state from the previous round to obtain a new hidden vector. The Actor network maps this vector to two-dimensional continuous coordinates and determines the specific patch vector index based on a pre-divided grid. This process is repeated four times: the first round coarsely locates the suspected lesion area; the second and third rounds perform finer-grained searches within local high-resolution regions; and the fourth round is used for error correction and supplementation. The selected patch vector is the image patch vector to be diagnosed.

[0080] More importantly, in multi-round indexing cycles, relying solely on the hidden state of the current round is susceptible to interference from local staining differences or mechanical noise, leading to decision drift. To address this, this embodiment employs FUSEX (Focused Ultrasound + EXcitation) to eliminate this disconnect through a three-step process: a memory buffer, multi-head cross-attention, and gated filtering. Specifically, the hidden vectors from the most recent rounds (default four rounds) are compressed to 256 dimensions and stored in a circular buffer. The current round's patch vector is used as the query vector, and multi-head attention matching is performed with the buffer entries to obtain a historical weighted summary. Finally, a sigmoid gate is used to adaptively allocate weights between the historical summary and the current patch vector, outputting a fused context-enhanced vector. This vector is used both to update the Critic and as the starting hidden state for the next round. Practice shows that after introducing FUSEX, the model is more sensitive to continuous morphological features across patches (such as CIN-level gradients and inflammatory margins), reducing the false negative rate by approximately 7%.

[0081] Preferably, the aggregator is used to assign attention weights to the image patch vector to be diagnosed, including:

[0082] From the iteration rounds corresponding to the image block vector to be diagnosed, extract candidate image block vectors and merge them with the image block vector to be diagnosed to obtain the feature package of that iteration round;

[0083] A two-layer nonlinear mapping network is used to compress the dimension of each vector in the feature bag and output attention weights.

[0084] The CLAM aggregator is introduced to further transform the high-value patch vectors selected by SmartPatchRL into a classification prediction and saliency heatmap of the entire WSI. CLAM achieves class discrimination and instance localization under weak supervision with only slide-level labels through a triple mechanism of "attention-gating-clustering". At the same time, its clustering constraints significantly improve feature discriminativeness and cross-batch robustness.

[0085] In attention weight allocation, firstly, the four key patch vectors output from step S3 are merged with the twelve candidate image patch vectors ranked highest in the same loop of attention, resulting in a feature pack with a length of no more than sixteen. Since redundant noise has been adequately filtered out in the early stages, this feature pack can significantly reduce memory consumption while maintaining the integrity of diagnostic information. Next, an attention score between 0 and 1 is calculated for each vector in the pack using a two-layer nonlinear mapping network: the first layer performs dimensionality compression and uses the tanh activation function to ensure gradient stability, while the second layer directly outputs normalized weights; higher weights indicate that the field of view is judged to have greater pathological value.

[0086] Preferably, an aggregator is used to perform clustering constraints based on the allocation results to obtain a global representation vector, including:

[0087] Based on the preset number of instance vectors, extract the vectors corresponding to the highest and lowest attention weights from the sorting of attention weights, respectively, and use them as potential positive examples and potential negative examples.

[0088] The potential positive examples and potential negative examples are respectively input into the clustering constraint branch to reduce the distance between vectors within a class and increase the distance between vectors between classes. The vectors in the feature bag after clustering constraint are weighted and summed according to the attention weights to output the global representation vector.

[0089] In this embodiment, the preset number of instance vectors is K, let K = 8, that is, 8 instances. After obtaining the attention ranking, the system automatically selects the k vectors with the highest weights as potential positive examples, and the k vectors with the lowest weights as potential negative examples. The vectors of potential positive examples and potential negative examples are respectively input into an additional clustering constraint branch: this branch reduces the Euclidean distance between vectors in the same group in the feature space and widens the distance between vectors in different groups, so that positive examples form close clusters and negative examples are far away from positive example clusters, thereby explicitly strengthening intra- and inter-class boundaries. In addition, the clustering constraint loss and the slice-level cross-entropy loss are weighted and combined in a ratio of 1:0.5 to jointly drive the end-to-end update of the network. To ensure smooth convergence, a two-stage strategy is preferred in the training phase: the feature encoding network is frozen for the first five epochs, and only the attention layer and clustering branch are trained. After the attention distribution stabilizes, the entire network is fine-tuned together until the twentieth epoch. The optimizer uses AdamW with weight decay, with the initial learning rate set to 2×10-4, and decreases according to the cosine curve after the linear warm-up in the first two epochs. By disabling the clustering branch during the visual encoder inference stage and retaining only the attention calculation and classification process, the average processing latency of a single slice does not exceed 0.3 seconds, which can meet the needs of clinical batch inference.

[0090] After completing the weight allocation and clustering constraints, the feature packet vectors are weighted and summed according to the attention weights to obtain a global representation vector that can represent the morphological information of the entire slice.

[0091] Preferably, the global representation vector is input into the classification head of the aggregator, and the lesion diagnosis result is output, including:

[0092] The global representation vector is input into the classification head of the aggregator to analyze and obtain multi-class diagnostic results and the probability of each class diagnostic result;

[0093] The diagnosis result corresponding to the category with the highest probability is used as the lesion diagnosis result and output.

[0094] Input the vector into the classification head, and output multiple diagnostic results such as "normal, low-grade lesion, high-grade lesion or cancer" and their corresponding probabilities, and take the category with the highest probability as the lesion diagnosis result.

[0095] Preferably, after outputting the lesion diagnosis results, the method further includes:

[0096] The attention weight of each vector in the feature package is mapped to the coordinate position of each vector in the full slice image, and an attention heatmap is generated based on the trajectory of the vector of the image block to be diagnosed.

[0097] After outputting the lesion diagnosis results, the attention weights of each image block vector to be diagnosed within the feature package are mapped back to their coordinate positions in the original whole-slice image. These weights are then overlaid on the slice thumbnail along with the four-round cyclic index trajectory to generate a saliency attention heatmap, which in turn outputs the lesion diagnosis results. This helps pathologists intuitively understand the model's focus areas and decision paths. It can also generate standardized case reports. When the classification model's confidence level falls below a set threshold, a pathology expert review mechanism is automatically triggered, and newly confirmed samples are added to the continuous learning sample pool to iteratively optimize the network, achieving proactive self-evolution of the diagnostic model.

[0098] Preferably, after obtaining the number of image patch vectors to be diagnosed corresponding to the number of iterations, the method further includes:

[0099] The image block vector to be diagnosed is input into a multilayer perceptron for diagnostic scoring, and the obtained score is used for policy update and advantage function calculation.

[0100] Each time the image patch vector to be diagnosed is selected in the indexing cycle, a diagnostic score is generated by a lightweight MLP (Multilayer Perceptron). This score serves as an immediate reward to drive the PPO policy update and is also written into the Critic evaluation network to provide the advantage function calculation. In this way, by using a continuous action space and leveraging reparameterized sampling, SmartPatchRL can maintain smooth policy gradients without introducing hard discrete indexing, resulting in fast and stable convergence during the indexing of the quality of the image patches to be diagnosed.

[0101] Throughout the training and analysis process, a three-loss joint optimization is employed: cross-entropy, PPO policy loss, and value regression. Cross-entropy accounts for the majority, ensuring classification performance; PPO loss uses a shearing coefficient to suppress policy oscillations; and the value regression term approximates the actual cumulative return for stable advantage estimation. The losses are weighted at 1:0.1:0.5 and backpropagated within the same computation graph. During the inference phase, only the feature matrix of the patch vector needs to be loaded once. SmartPatchRL completes four rounds of chip selection in GPU memory, with an average of 90ms per loop; FUSEX's three historical fusions take approximately 200ms; the overall S20 stage single-WSI inference latency does not exceed 1.2s, and it supports automatic dimensionality reduction and CPU fallback when GPU memory is insufficient, ensuring that >2500 WSIs can be processed daily on a single 24GB GPU. Figure 4 The training involved models and their parameter configurations are shown in the table.

[0102] This method balances diagnostic sensitivity, structural integrity, and clinical interpretability when diagnosing whole-section images of cervical tissue. Figure 5As shown, the accuracy results of this method compared to other commonly used MIL (Multiple Instance Learning) classification tasks are illustrated, with bold indicating the best, ^ indicating the second best, and * indicating the third best. For pathological diagnosis of whole-slice images, it demonstrates superior performance across multiple key evaluation metrics. Specifically, the accuracy reaches 91.42%, the Macro F1-score reaches 91.61%, and the Macro AUC reaches 98.73%, ranking first among all compared models. This not only demonstrates the model's excellent overall classification accuracy but also further confirms its strong advantages in class balance and generalization ability. In contrast, other better-performing models such as AMD_MIL and WIKG_MIL, while approaching or entering the top three in some metrics, still lag behind this model in overall performance, especially in terms of stability and peak performance across the three metrics, where this application significantly outperforms. In summary, this study significantly improves classification accuracy and lesion localization reliability while maintaining high sensitivity. It demonstrates high detection accuracy and robustness for cervical cancer and precancerous lesions, providing reliable data support and technical tools for pathological diagnosis, research, and teaching. It offers clinicians a trustworthy basis for lesion grade determination and surgical boundary assessment. For female reproductive systems other than the cervix, minor adjustments can be made to transfer the data to related diseases. Furthermore, it has the capability to process tens of thousands of high-resolution slides at high speed, supporting large-scale cohort studies and data anonymization sharing in regional pathology centers and research institutions. In addition, the automatically generated saliency heatmaps and multi-scale features can be used to explore the progression path of precancerous lesions and changes in the microenvironment, promoting in-depth basic medical research.

[0103] Example 2

[0104] Based on the same principle as the aforementioned methods, a cervical tissue section lesion diagnostic device 100 based on reinforcement learning is also proposed, such as... Figure 6 As shown, it includes:

[0105] The image block acquisition module 110 is used to crop the full slice image of the standardized cervical tissue slice to be diagnosed according to the preset cropping parameters, and obtain image blocks corresponding to different cropping parameters.

[0106] The feature analysis module 120 is used to input the image block into the visual encoder, and the visual encoder performs feature reasoning analysis on the image block to obtain the features of the whole slice image.

[0107] The indexing module 130 is used to calculate the hidden vector of the feature using a reinforcement learning model, and to perform indexing based on the two-dimensional continuous coordinates corresponding to the hidden vector for a preset number of iterations to obtain the image patch vector to be diagnosed.

[0108] The diagnostic module 140 is used to allocate attention weights to the image block vector to be diagnosed using an aggregator, and to perform clustering constraints based on the allocation results to obtain a global representation vector. The global representation vector is then input into the classification head of the aggregator to output the lesion diagnosis result.

[0109] Example 3

[0110] Furthermore, an electronic device is proposed, comprising:

[0111] processor;

[0112] Memory used to store processor-executable instructions;

[0113] The processor is configured to implement the reinforcement learning-based cervical tissue section lesion diagnosis method described in Embodiment 1 when executing the executable instructions.

[0114] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0115] Furthermore, it should be noted that the use of terms such as "first," "second," and "a" in this invention is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. The terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two elements or the interaction between two elements, unless otherwise explicitly specified. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0116] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0117] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for diagnosing cervical tissue section lesions based on reinforcement learning, characterized in that, include: The full-slice image of the cervical tissue slice to be diagnosed is cropped according to the preset cropping parameters to obtain image blocks corresponding to different cropping parameters; The image patch is input into a visual encoder, which performs feature reasoning analysis on the image patch to obtain the features of the full slice image. The hidden vector of the feature is calculated using a reinforcement learning model, and the image patch vector to be diagnosed is obtained by indexing the two-dimensional continuous coordinates corresponding to the hidden vector for a preset number of iterations. An aggregator is used to assign attention weights to the image patch vectors to be diagnosed, and clustering constraints are applied based on the assignment results to obtain a global representation vector. The global representation vector is then input into the classification head of the aggregator to output the lesion diagnosis result.

2. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 1, characterized in that, The image patch is input into a visual encoder, which performs feature inference analysis on the image patch to obtain the features of the whole slice image, including: The image blocks are input into the visual encoder according to the preset batch size, and batch inference is performed in a gradient-free environment to obtain the global generalization vector of the whole slice image and the local average vector of the image blocks.

3. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 1, characterized in that, The hidden vector of the feature is calculated using a reinforcement learning model. Based on the two-dimensional continuous coordinates corresponding to the hidden vector, an index is applied for a preset number of iterations to obtain the image patch vector to be diagnosed, including: The features are averaged and normalized, and then combined with the hidden state of the previous iteration and input into the gated recurrent unit to obtain the hidden vector corresponding to the features. The hidden vectors are mapped to two-dimensional continuous coordinates using an Actor network; The image block vector to be diagnosed is indexed according to the pre-divided grid and the two-dimensional continuous coordinates. Based on the indexing result of each time, it is indexed again according to the preset number of iterations to obtain the number of image block vectors to be diagnosed corresponding to the number of iterations.

4. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 3, characterized in that, The aggregator is used to assign attention weights to the image patch vector to be diagnosed, including: From the iteration rounds corresponding to the image block vector to be diagnosed, extract candidate image block vectors and merge them with the image block vector to be diagnosed to obtain the feature package of that iteration round; A two-layer nonlinear mapping network is used to compress the dimension of each vector in the feature bag and output attention weights.

5. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 4, characterized in that, The aggregator is used to apply clustering constraints based on the assignment results to obtain a global representation vector, including: Based on the preset number of instance vectors, extract the vectors corresponding to the highest and lowest attention weights from the sorting of attention weights, respectively, and use them as potential positive examples and potential negative examples. The potential positive examples and potential negative examples are respectively input into the clustering constraint branch to reduce the distance between vectors within a class and increase the distance between vectors between classes. The vectors in the feature bag after clustering constraint are weighted and summed according to the attention weights to output the global representation vector.

6. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 1, characterized in that, The global representation vector is input into the classification head of the aggregator, and the lesion diagnosis results are output, including: The global representation vector is input into the classification head of the aggregator to analyze and obtain multi-class diagnostic results and the probability of each class diagnostic result; The diagnosis result corresponding to the category with the highest probability is used as the lesion diagnosis result and output.

7. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 5, characterized in that, The output of the lesion diagnosis results also includes: The attention weight of each vector in the feature package is mapped to the coordinate position of each vector in the full slice image, and an attention heatmap is generated based on the trajectory of the vector of the image block to be diagnosed.

8. The method for diagnosing cervical tissue section lesions based on reinforcement learning according to claim 3, characterized in that, After obtaining the number of image patch vectors to be diagnosed corresponding to the number of iterations, the method further includes: The image block vector to be diagnosed is input into a multilayer perceptron for diagnostic scoring, and the obtained score is used for policy update and advantage function calculation.

9. A cervical tissue section lesion diagnostic device based on reinforcement learning, characterized in that, include: The image block acquisition module is used to crop the full-slice image of the standardized cervical tissue slice to be diagnosed according to preset cropping parameters, and obtain image blocks corresponding to different cropping parameters; The feature analysis module is used to input the image patch into the visual encoder, and the visual encoder performs feature reasoning analysis on the image patch to obtain the features of the whole slice image. The indexing module is used to calculate the hidden vector of the feature using a reinforcement learning model, and to perform indexing based on the two-dimensional continuous coordinates corresponding to the hidden vector for a preset number of iterations to obtain the image patch vector to be diagnosed. The diagnostic module is used to assign attention weights to the image block vector to be diagnosed using an aggregator, and to perform clustering constraints based on the assignment results to obtain a global representation vector. The global representation vector is then input into the classification head of the aggregator to output the lesion diagnosis result.

10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the reinforcement learning-based cervical tissue section lesion diagnosis method according to any one of claims 1-8 when executing the executable instructions.

Images