Cancer grade prediction method, device, equipment and storage medium

Abstract

The present invention discloses a method, apparatus, device, and storage medium for predicting cancer grade, comprising: collecting a patient's pathological images and their corresponding grade annotation information, and preprocessing them to divide them into a number of image blocks; using the image blocks of each pathological image to train a self-supervised learning model; clustering the image block coding vectors using a clustering algorithm to cluster the image block coding vectors of each pathological image into a number of category regions; sampling features of the image block coding vectors of each category region, and fusing the features of the sampled image block coding vectors to obtain image fusion features of each pathological image; using the image fusion features with category region information of each pathological image to train a cancer grade model, the cancer grade model being used to predict cancer grade; and applying the trained model to the pathological image to be analyzed to predict cancer grade. The present invention can accurately grade cancer, facilitating timely disease detection and the development of treatment plans.

Description

Technical Field

[0001] The present invention belongs to the technical field of pathological image processing, and specifically relates to a cancer grading prediction method, device, equipment and storage medium. Background Art

[0002] Cervical histopathology is the gold standard for diagnosing cervical cancer. Doctors examine the cell morphology and structure of pathological tissue sections under a microscope to determine the grade of cervical cancer. However, pathological diagnosis results are often affected by multiple subjective factors, and the process of examining cell and tissue morphology field by field is time-consuming and labor-intensive, resulting in low efficiency and a high reliance on human factors and subjective judgment. The application of artificial intelligence (AI) in cancer grade prediction is rapidly developing, primarily through data mining, machine learning, and deep learning. However, existing AI prediction methods have the following issues.

[0003] First, existing pathology image segmentation methods rely on manual annotation and a Unet neural network to segment tumor regions in low-resolution pathology images. However, cervical squamous cell carcinoma tumor regions are difficult to capture in low-resolution images, failing to meet accuracy requirements.

[0004] Second, the pixel level of fully digital pathology images ranges from tens of thousands to hundreds of thousands, making image processing difficult and time-consuming.

[0005] Third, tumor cells acquire the ability to destroy surrounding tissues through invasive growth. Therefore, the grade of the tumor is closely related to the degree of invasion of the tumor into adjacent tissues. Existing grading methods do not consider the connection between regions. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention proposes a cancer grade prediction method, device, equipment and storage medium.

[0007] In order to achieve the above object, the technical solution of the present invention is as follows:

[0008] In a first aspect, the present invention discloses a method for predicting cancer grade, comprising:

[0009] Step S1: Collect the patient's pathological images and their corresponding graded annotation information;

[0010] Step S2: preprocessing each collected pathological image;

[0011] Step S3: Divide each preprocessed pathological image into several image blocks;

[0012] Step S4: using the image blocks of each pathological image obtained in step S3 to train a self-supervised learning model, the self-supervised learning model is used to divide each image block into a number of image slices, and extract image slice features of each image slice to obtain an image block encoding vector for each image block;

[0013] Step S5: clustering the image block coding vectors obtained in step S4 using a clustering algorithm, clustering the image block coding vectors of each pathological image into several category regions;

[0014] Step S6: performing feature sampling on the image block coding vectors of each category area, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of each pathological image;

[0015] Step S7: using the image fusion features with category region information of each pathological image obtained in step S6 to train a cancer grading model, the cancer grading model is used to predict cancer grade;

[0016] Step S8: Apply the trained model to the pathological image to be analyzed to predict the cancer grade.

[0017] On the basis of the above technical solution, the following improvements can be made:

[0018] As a preferred solution, the preprocessing operations in step S2 include: removing image noise, removing image artifacts, removing image background, and data augmentation.

[0019] As a preferred solution, step S6 includes:

[0020] Step S6.1: performing feature sampling on the image block coding vectors of each category region of each pathological image, and selecting representative image block coding vectors in each category region;

[0021] Step S6.2: Perform feature concatenation on all the image block encoding vectors within each category region after sampling to obtain the regional fusion features of each category region;

[0022] Step S6.3: Perform feature splicing on the regional fusion features of all category regions of each pathological image to obtain the image fusion features of each pathological image.

[0023] As a preferred solution, step S8 includes:

[0024] Step S8.1: Collect the patient's pathological images to be analyzed;

[0025] Step S8.2: pre-processing the collected pathological images to be analyzed;

[0026] Step S8.3: Divide the preprocessed pathological image into a number of image blocks;

[0027] Step S8.4: Divide the image block of the pathological image to be analyzed obtained in step S8.3 into a number of image slices using the trained self-supervised learning model, extract the image slice features of each image slice, and obtain the image block encoding vector of each image block;

[0028] Step S8.5: clustering the image block coding vectors obtained in step S8.4 using a clustering algorithm, clustering the image block coding vectors of the pathological image to be analyzed into several category regions;

[0029] Step S8.6: performing feature sampling on the image block coding vectors of each category region, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of the pathological image to be analyzed;

[0030] Step S8.7: Based on the image fusion features of the pathological image to be analyzed obtained in step S8.6, the patient's cancer grade is predicted using the trained cancer grade model.

[0031] In a second aspect, the present invention further discloses a cancer grade prediction device, comprising:

[0032] A collection module is used to collect the patient's pathological images and their corresponding graded annotation information;

[0033] A preprocessing module, used to preprocess each collected pathological image;

[0034] A division module, used for dividing each preprocessed pathological image into a number of image blocks;

[0035] a self-supervised learning model training module, configured to train a self-supervised learning model using the image blocks of each pathological image obtained by the segmentation module, wherein the self-supervised learning model is configured to segment each image block into a plurality of image slices, extract image slice features of each image slice, and obtain an image block encoding vector for each image block;

[0036] A clustering module is used to cluster the image block coding vectors obtained by the self-supervised learning model training module using a clustering algorithm, and cluster the image block coding vectors of each pathological image into several category regions;

[0037] A sampling and fusion module is used to perform feature sampling on the image block coding vectors of each category area and perform feature fusion on the sampled image block coding vectors to obtain image fusion features of each pathological image;

[0038] a cancer grading model training module, configured to train a cancer grading model using the image fusion features with category region information of each pathological image obtained by the sampling fusion module, wherein the cancer grading model is used to predict cancer grading;

[0039] The application module is used to apply the trained model to the pathological images to be analyzed and predict cancer grade.

[0040] As a preferred solution, the preprocessing operations in the preprocessing module include: removing image noise, removing image artifacts, removing image background, and data augmentation.

[0041] As a preferred solution, the sampling fusion module includes:

[0042] A sampling unit, configured to perform feature sampling on the image block coding vectors of each category region of each pathological image, and select representative image block coding vectors in each category region;

[0043] The first fusion unit is used to perform feature splicing on all the image block coding vectors in each category area after sampling to obtain the regional fusion features of each category area;

[0044] The second fusion unit is used to perform feature splicing on the regional fusion features of all category regions of each pathological image to obtain the image fusion features of each pathological image.

[0045] As a preferred solution, the application modules include:

[0046] An application collection unit is used to collect the patient's pathological images to be analyzed;

[0047] An application preprocessing unit is used to preprocess the collected pathological images to be analyzed;

[0048] Applying a division unit, for dividing the preprocessed pathological image into a plurality of image blocks;

[0049] an application feature extraction unit, configured to divide the image block of the pathological image to be analyzed obtained by the application division unit into a plurality of image slices using the trained self-supervised learning model, and extract image slice features of each image slice to obtain an image block coding vector for each image block;

[0050] An application clustering unit is used to cluster the image block coding vectors obtained by the application feature extraction unit using a clustering algorithm, and cluster the image block coding vectors of the pathological image to be analyzed into a plurality of category regions;

[0051] Applying a sampling and fusion unit to perform feature sampling on the image block coding vectors of each category area, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of the pathological image to be analyzed;

[0052] The application grading unit is used to predict the cancer grade of the patient using the trained cancer grading model based on the image fusion features of the pathological image to be analyzed obtained by the application sampling fusion unit.

[0053] In a third aspect, the present invention further discloses a computing device, comprising:

[0054] one or more processors;

[0055] Memory;

[0056] and one or more programs, wherein the one or more programs are stored in a memory and configured to be executed by one or more processors, and the one or more programs include instructions of any of the above cancer grade prediction methods.

[0057] In a fourth aspect, the present invention further discloses a storage medium storing one or more computer-readable programs, wherein the one or more programs include instructions suitable for being loaded by a memory and executing any of the above-mentioned cancer grading prediction methods.

[0058] The present invention discloses a cancer grade prediction method, device, equipment and storage medium, which have the following beneficial effects:

[0059] First, the present invention uses self-supervised learning technology to enhance the model's ability to represent pathological images and reduce its dependence on labeled data, thereby improving the model's generalization ability.

[0060] Second, the present invention uses a feature clustering algorithm to distinguish key areas in pathology images, which can speed up subsequent computations and save time. The clustering algorithm can handle different staining styles or cancer types based on the specific conditions of the pathology images, enhancing the algorithm's generalization capabilities.

[0061] Third, the present invention performs feature sampling and feature fusion on different features after clustering, which can avoid the situation where the prediction bias is caused by the small proportion of tumor tissue.

[0062] Fourth, the present invention trains a cancer grading model by fusing features of images rich in information from different category regions. The degree of tumor grading may be related to the degree of canceration in the tissue adjacent to the tumor. The cancer grading model based on category regions can capture subtle changes in pathological images, including cell morphology, structural heterogeneity, and characteristics of the tumor microenvironment, and takes into account the influence between different category regions and the influence within the category regions.

[0063] In summary, the present invention proposes a cancer grading prediction method based on self-supervised learning and feature clustering, which can accurately grade cancer, help to timely detect the disease and formulate treatment plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0065] Figure 1 This is a flowchart of a cancer grade prediction method provided by an embodiment of the present invention.

[0066] Figure 2 A schematic diagram of the process of feature extraction of a self-supervised learning model provided by an embodiment of the present invention.

[0067] FIG3( a ) is an original pathological image provided by an embodiment of the present invention;

[0068] FIG3( b ) is a diagram showing the clustering results provided by an embodiment of the present invention.

[0069] Figure 4 A schematic diagram of the process of cancer grading model prediction provided by an embodiment of the present invention.

[0070] Figure 5 This is the AUROC curve provided by an embodiment of the present invention.

[0071] Figure 6 A schematic diagram of the application process of an embodiment of the present invention.

[0072] Figure 7 This is a block diagram of a cancer grade prediction device provided by an embodiment of the present invention.

[0073] Figure 8 A block diagram of a computing device provided in accordance with an embodiment of the present invention.

[0074] Among them: 201-collection module, 202-preprocessing module, 203-partitioning module, 204-self-supervised learning model training module, 205-clustering module, 206-sampling fusion module, 207-cancer grading model training module, 208-application module, 301-processor, 302-memory. DETAILED DESCRIPTION

[0075] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0076] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0077] The use of ordinal numbers "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are involved and is not intended to imply that the objects so described must have a given order in time, space, ranking, or in any other manner.

[0078] In addition, the expression of “comprising” an element is an “open” expression, which merely means that corresponding components or steps exist, and should not be interpreted as excluding additional components or steps.

[0079] In order to achieve the purpose of the present invention, some embodiments of the cancer grade prediction method take cervical cancer as an example. Figure 1 As shown, the cancer grade prediction methods include:

[0080] Step S101: collecting the patient's pathological images and their corresponding graded annotation information;

[0081] Step S102: pre-processing each collected pathological image;

[0082] Step S103: Divide each pre-processed pathological image into a number of image blocks;

[0083] Step S104: using the image blocks of each pathological image obtained in step S103 to train a self-supervised learning model, the self-supervised learning model is used to divide each image block into a number of image slices, and extract image slice features of each image slice to obtain an image block encoding vector for each image block;

[0084] Step S105: clustering the image block coding vectors obtained in step S104 using a clustering algorithm, and clustering the image block coding vectors of each pathological image into several category regions;

[0085] Step S106: performing feature sampling on the image block coding vectors of each category area, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of each pathological image;

[0086] Step S107: using the image fusion features with category region information of each pathological image obtained in step S106 to train a cancer grading model, the cancer grading model is used to predict cancer grade;

[0087] Step S108: Apply the trained model to the pathological image to be analyzed to predict the cancer grade.

[0088] Each of the above steps is explained in detail below.

[0089] Step S101 collects whole slide pathology images (WSI) of patients with cervical squamous cell carcinoma.

[0090] Specifically, in this example, the IMP-cervix dataset was used for experiments to construct the cervical squamous cell carcinoma dataset of the present invention. The IMP-cervix dataset is a medical image dataset specifically used to study cervical lesions. It contains 600 cervical LEEP (loop electrosurgical excision procedure) samples and surgical specimens, which were obtained from the data archives of the IMP Diagnostic Laboratory in Portugal. The samples were digitized using two Leica GT450 whole slide image (WSI) scanners, with a scan magnification of 40x and a pixel size of 0.26 microns. The samples can be specifically divided into the following four categories:

[0091] 1) Non-neoplastic (NNeo, grade 0): normal cervical tissue, no lesions.

[0092] 2) Low-grade squamous intraepithelial lesion (LSIL, grade 1): mild cervical lesions.

[0093] 3) High-grade squamous intraepithelial lesions (HSIL, grade 2): moderate to severe cervical lesions.

[0094] 4) Non-representative (other, category 3): including other atypical samples.

[0095] In this example, the cervical squamous cell carcinoma dataset is divided into a training set and a test set in a ratio of 6:4. The training set is used for model training, and the test set is used to evaluate the performance of the model.

[0096] like Figure 2 As shown, step S102 preprocesses each collected pathology image, including but not limited to: removing image noise, removing image artifacts, removing image background, and performing data augmentation. By rotating, flipping, and cropping the original pathology images, a series of new pathology images are generated, which increases the amount of training data for subsequent models. This can effectively expand the dataset, enable the model to better adapt to different pathology image distributions, and improve classification accuracy and stability.

[0097] Define the size of layer 0 of the pathology image (high width), set the zoom ratio to B, then the size of the thumbnail is .

[0098] The threshold segmentation method is used to separate the tissue and background of the thumbnail to obtain a tissue mask image. Subsequently, the corresponding image blocks are derived based on the tissue mask image, which can effectively reduce redundant background image blocks.

[0099] The threshold segmentation formula is as follows:

[0100]

[0101] in: Represents the tissue mask image, R, G, B represent the RGB three-channel values ​​of the thumbnail.

[0102] Of course, in some other embodiments, a deep learning model with a lightweight network structure, such as MobileNet or ShuffleNet, may be used for tissue segmentation.

[0103] Step S103 divides each pre-processed pathological image into a number of image tiles.

[0104] Defines the physical size of the image block ,in: is the length of the image block, is the width of the image block.

[0105] The conversion formula between physical size and pixel points is as follows:

[0106]

[0107] Among them: mpp is the inherent attribute of the pathological image, Represents a pixel.

[0108] According to the tissue mask image and Export tissue image blocks. The number of image blocks exported for each pathology image varies depending on the physical size and tissue mask image.

[0109] Taking a pathological image as an example, define =256um, =256um, 1724 image blocks can be obtained, and the size of the image blocks is 512×512. express, , N represents the number of image blocks, represents the i-th image block.

[0110] Step S104 trains the self-supervised learning model.

[0111] Specifically, in this embodiment, the self-supervised learning model is trained using image blocks from the training set of the cervical squamous cell carcinoma dataset, using the Dinov2 model as the backbone network and saving the optimal model. The self-supervised learning model can efficiently extract multi-level pathological features from pathological images.

[0112] Feature extraction for self-supervised learning models includes:

[0113] First, the image block is scaled by the scaling function to obtain the scaled image block , the scaled image block size is 224×224.

[0114]

[0115] Rescale represents a scaling function, which can be a method such as bilinear interpolation or nearest neighbor interpolation. In this example, bilinear interpolation is used for scaling.

[0116] Rescaled image blocks Input into the pre-trained model to extract features. The pre-trained model divides the image block into non-overlapping image patches, flattens each image patch into a vector, and generates the initial feature representation through mapping:

[0117]

[0118] in: is the flattening function, Indicates the images of is the linear mapping matrix, is the bias term.

[0119] The pre-trained model considers the relative position between each image slice and the feature vector of each image slice. Perform position encoding:

[0120]

[0121] in, is the position code, and M is the number of image slices.

[0122] Feature vector after position encoding Input into the multi-layer Transformer and obtain the encoding vector E by aggregating in the multi-layer attention mechanism. .

[0123]

[0124] Where: E represents the image block encoding vector, L represents the feature dimension, MAS represents the multi-head attention mechanism, Indicates the number of layers of Transformer.

[0125] Self-supervised learning can extract useful feature representations from unlabeled data. Self-supervised learning models can learn effective features even with only a small amount of labeled data.

[0126] Step S105 clusters the image block encoding vectors obtained using a clustering algorithm, which may be, but is not limited to, K-Means, hierarchical clustering, DBSCAN, and the like.

[0127] Specifically, in this embodiment, taking the K-Means algorithm as an example, the number of clusters k is set to 4, and for each image block encoding vector (where i=1,2,...,N), calculate its distance to each cluster center and assign it to the nearest cluster center. Use the following formula to calculate the distance:

[0128]

[0129] Where: d represents the distance calculation formula, represents the j-th cluster center.

[0130] By clustering, a feature set is formed .

[0131] in: It is a set of image block encoding vectors of 4 category areas, representing the 4 category areas of the pathological image.

[0132] Through the above clustering operation, the image blocks of the pathological image can be clustered into four category regions. Each category region is represented by a set of image block encoding vectors, which can be used to further analyze and understand the structure and characteristics of the pathological image.

[0133] The present invention uses a feature clustering algorithm to distinguish the main areas of cervical squamous cell carcinoma pathology images. The clustering algorithm can handle different staining styles or different cancer types according to the actual conditions of the pathology images, thereby enhancing the generalization performance of the algorithm.

[0134] Fully digital pathology images have pixels in the hundreds of thousands or even larger. Existing methods struggle to meet precision requirements for identifying regions of interest at low resolutions. Feature clustering addresses these challenges, improving processing speed and accuracy, while accelerating computation and saving time and costs.

[0135] Figure 3(a) is the original pathological image, and Figure 3(b) shows the pathological images of the four category areas after clustering.

[0136] Step S106 performs feature sampling and feature fusion on the image block coding vectors of each category area to obtain the image fusion features of each pathological image. Specifically, it includes:

[0137] Step S106.1: Perform feature sampling on the image block coding vectors of each category region of each pathological image, and select representative image block coding vectors in each category region;

[0138] Step S106.2: Perform feature concatenation on all the image block coding vectors within each category region after sampling to obtain a regional fusion feature for each category region;

[0139] Step S106.3: performing feature splicing on the regional fusion features of all category regions of each pathological image to obtain the image fusion features of each pathological image.

[0140] Specifically, in this embodiment, a random sampling strategy is adopted to obtain representative samples from the feature set F. The screening rules are set as follows:

[0141]

[0142] in: represents the number of samples, Indicates the number of features in each category area, min represents the minimum function. Follow the above rules to obtain the feature set after sampling }.

[0143] The vectors in the sampled feature set are concatenated to obtain the sampled image fusion feature F_concat. , S represents the number of features after splicing.

[0144] The present invention randomly extracts different features after clustering to form new features, which can avoid the situation where the prediction bias is caused by the small proportion of tumor tissue. Randomly extracting different regional features after clustering can ensure that the tumor tissue and other tissues are analyzed at the same quantity level, and also avoid the problem of tumor tissue screening.

[0145] The existing method extracts features from image blocks of the entire digital slide image and inputs them into the model, randomly extracting samples from different areas, which speeds up the training time of subsequent tasks. The extracted clustered sample features ensure the richness of the pathological image and avoid the problem of data omission.

[0146] like Figure 4 As shown, step S107 uses the obtained image fusion features to train a cancer grading model. The trained cancer grading model is a regression model based on different category regions, which consists of an encoder, an attention mechanism, and a final scoring mechanism. The specific structure is as follows.

[0147] The encoder converts the image fusion feature F_concat into a low-dimensional vector:

[0148]

[0149] in, Represents vector dot multiplication operation, represents the weight matrix, represents the bias term, Represents the activation function. It is the aggregate feature representation of each category area.

[0150] The attention mechanism is used to calculate the importance weights of category regions in order to weight instance features in subsequent steps. It calculates the importance scores by mapping features to smaller dimensions.

[0151] The calculation of the attention score involves three parts: linear transformation, activation function, and output layer. The linear transformation maps the input features to low dimensions, and after the linear transformation, the linear transformation is input to the activation function. The formula is as follows:

[0152]

[0153] in: represents the activation function, is the linear transformation matrix, Is the bias matrix, which activates the output of the function Enter another linear change layer to get the final attention score .

[0154]

[0155] in: Represents the weight matrix. The attention mechanism performs masking based on the number (length) of instances to prevent weight calculation for empty instances.

[0156] Masked attention scores Normalized by the Softmax function to ensure that the sum of the weights is 1:

[0157]

[0158] The weights obtained by the attention mechanism are weighted and summed to generate a global weighted feature representation. :

[0159]

[0160] in: represents the i-th encoding feature, is the attention score corresponding to the i-th encoded feature. The output layer is used to perform the final score on the weighted global weighted features to obtain the final score of cervical squamous cell carcinoma.

[0161] The output layer contains a linear layer, and the output calculation formula is:

[0162]

[0163] In the above formula is the weight matrix of the output layer, is the bias term, It is the prediction score of the model, and reasonable classification rules are formulated based on the prediction score.

[0164] In this example, the score is set between 0 and 1 to represent level 0 (NNeo), between 1 and 2 to represent level 1 (LSIL), between 2 and 3 to represent level 3 (HSIL), and between 3 and 4 to represent non-representative. MSE is used as the loss function for gradient update, and the formula is as follows:

[0165]

[0166] Where: n is the number of samples, Indicates the The actual value of the samples, Indicates the The predicted value of the sample.

[0167] Save the model after 25 iterations.

[0168] like Figure 5 As shown in the figure, in this example, AUROC was used to evaluate the test set and identify the optimal model for clinical application. AUROC (Area Under the Receiver Operating Characteristic Curve) is the area under the ROC curve. The cancer grading model ultimately achieved an AUROC of 0.8998 for predicting grade 0, 0.8306 for predicting grade 1, 0.9066 for predicting grade 2, and 0.9766 for predicting grade 3.

[0169] The present invention trains a cancer grading model by fusing features of images rich in information from different category regions. The degree of tumor grading may be related to the degree of canceration in the tissue adjacent to the tumor. The cancer grading model based on category regions can capture subtle changes in pathological images, including cell morphology, structural heterogeneity, and characteristics of the tumor microenvironment, and takes into account the influence between different category regions and the influence within the category regions.

[0170] By using a regression algorithm to perform detailed analysis and scoring of each category region in a pathology image, the cancer grading model can adapt to different pathology images and grading requirements. By adjusting the regression algorithm parameters and grading rules, it can flexibly handle different types of pathology images. This adaptability allows the model to be applied not only to a single pathology type but also to a wider range of scenarios.

[0171] like Figure 6 As shown, step S108 is an application step, including:

[0172] Step S108.1: Collect the patient's pathological images to be analyzed;

[0173] Step S108.2: pre-processing the collected pathological images to be analyzed;

[0174] Step S108.3: Divide the preprocessed pathological image into a number of image blocks;

[0175] Step S108.4: Using the trained self-supervised learning model, the image block of the pathological image to be analyzed obtained in step S8.3 is divided into a number of image slices, and image slice features of each image slice are extracted to obtain an image block encoding vector for each image block;

[0176] Step S108.5: clustering the image block coding vectors obtained in step S8.4 using a clustering algorithm, clustering the image block coding vectors of the pathological image to be analyzed into several category regions;

[0177] Step S108.6: performing feature sampling on the image block coding vectors of each category region, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of the pathological image to be analyzed;

[0178] Step S108.7: Based on the image fusion features of the pathological image to be analyzed obtained in step S108.6, the patient's cancer grade is predicted using the trained cancer grade model.

[0179] After the experiment, the patient's prediction score was 2.68 and the predicted grade was 3 (HSIL).

[0180] In addition, in some other embodiments, Figure 7 As shown, the present invention also discloses a cancer grade prediction device, comprising:

[0181] The collection module 201 is used to collect the patient's pathological images and their corresponding graded annotation information;

[0182] A preprocessing module 202 is used to preprocess each collected pathological image;

[0183] A division module 203 is used to divide each pre-processed pathological image into a number of image blocks;

[0184] A self-supervised learning model training module 204 is used to train a self-supervised learning model using the image blocks of each pathological image obtained by the segmentation module. The self-supervised learning model is used to divide each image block into a number of image slices, extract image slice features of each image slice, and obtain an image block coding vector for each image block;

[0185] A clustering module 205 is configured to cluster the image block coding vectors obtained by the self-supervised learning model training module using a clustering algorithm, and cluster the image block coding vectors of each pathological image into a plurality of category regions;

[0186] A sampling and fusion module 206 is used to perform feature sampling on the image block coding vectors of each category area and perform feature fusion on the sampled image block coding vectors to obtain image fusion features of each pathological image;

[0187] a cancer grading model training module 207, configured to train a cancer grading model using the image fusion features with category region information of each pathological image obtained by the sampling fusion module, wherein the cancer grading model is used to predict cancer grading;

[0188] The application module 208 is used to apply the trained model to the pathological image to be analyzed to predict the cancer grade.

[0189] Furthermore, the preprocessing operations in the preprocessing module include: removing image noise, removing image artifacts, removing image background, and data augmentation.

[0190] Furthermore, the sampling fusion module includes:

[0191] A sampling unit, configured to perform feature sampling on the image block coding vectors of each category region of each pathological image, and select representative image block coding vectors in each category region;

[0192] The first fusion unit is used to perform feature splicing on all the image block coding vectors in each category area after sampling to obtain the regional fusion features of each category area;

[0193] The second fusion unit is used to perform feature splicing on the regional fusion features of all category regions of each pathological image to obtain the image fusion features of each pathological image.

[0194] Furthermore, the application module includes:

[0195] An application collection unit is used to collect the patient's pathological images to be analyzed;

[0196] An application preprocessing unit is used to preprocess the collected pathological images to be analyzed;

[0197] Applying a division unit, for dividing the preprocessed pathological image into a plurality of image blocks;

[0198] an application feature extraction unit, configured to divide the image block of the pathological image to be analyzed obtained by the application division unit into a plurality of image slices using the trained self-supervised learning model, and extract image slice features of each image slice to obtain an image block coding vector for each image block;

[0199] An application clustering unit is used to cluster the image block coding vectors obtained by the application feature extraction unit using a clustering algorithm, and cluster the image block coding vectors of the pathological image to be analyzed into a plurality of category regions;

[0200] Applying a sampling and fusion unit to perform feature sampling on the image block coding vectors of each category area, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of the pathological image to be analyzed;

[0201] The application grading unit is used to predict the cancer grade of the patient using the trained cancer grading model based on the image fusion features of the pathological image to be analyzed obtained by the application sampling fusion unit.

[0202] Furthermore, it should be noted that the cancer grading prediction device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example when performing grading prediction. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the cancer grading prediction device can be divided into different functional modules to complete all or part of the functions described above.

[0203] In addition, the cancer grade prediction device and the cancer grade prediction method provided in the above embodiments belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0204] In addition, in some other embodiments, Figure 8 As shown, the present invention also discloses a computing device, including:

[0205] One or more processors 301;

[0206] Memory 302;

[0207] and one or more programs, wherein the one or more programs are stored in the memory 302 and configured to be executed by the one or more processors 301 , and the one or more programs include instructions of the cancer grade prediction method disclosed in the above embodiment.

[0208] The processor 301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 301 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 301 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 301 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0209] Memory 302 may include one or more computer-readable storage media, which may be non-transitory. Memory 302 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in memory 302 is used to store at least one instruction, which is executed by processor 301 to implement the cancer grade prediction method provided by the method embodiment of the present invention.

[0210] In addition, the computing device may optionally include a peripheral device interface and at least one peripheral device. The processor 301, memory 302, and peripheral device interface may be connected via a bus or signal lines. Each peripheral device may be connected to the peripheral device interface via a bus, signal lines, or circuit boards. Illustratively, the peripheral devices include, but are not limited to, radio frequency circuitry, a touchscreen display, audio circuitry, and a power supply.

[0211] Of course, the computing device may also include fewer or more components, which is not limited in this embodiment.

[0212] In addition, in some other embodiments, the present invention further discloses a storage medium storing one or more computer-readable programs, wherein the one or more programs include instructions suitable for being loaded by a memory and executing the cancer grading prediction method disclosed in the above embodiment.

[0213] The present invention discloses a cancer grade prediction method, device, equipment and storage medium, which have the following beneficial effects:

[0214] First, the present invention uses self-supervised learning technology to enhance the model's ability to represent pathological images and reduce its dependence on labeled data, thereby improving the model's generalization ability.

[0215] Second, the present invention uses a feature clustering algorithm to distinguish key areas in pathology images, which can speed up subsequent computations and save time. The clustering algorithm can handle different staining styles or cancer types based on the specific conditions of the pathology images, enhancing the algorithm's generalization capabilities.

[0216] Third, the present invention performs feature sampling and feature fusion on different features after clustering, which can avoid the situation where the prediction bias is caused by the small proportion of tumor tissue.

[0217] Fourth, the present invention trains a cancer grading model by fusing features of images rich in information from different category regions. The degree of tumor grading may be related to the degree of canceration in the tissue adjacent to the tumor. The cancer grading model based on category regions can capture subtle changes in pathological images, including cell morphology, structural heterogeneity, and characteristics of the tumor microenvironment, and takes into account the influence between different category regions and the influence within the category regions.

[0218] In summary, the present invention proposes a cancer grading prediction method based on self-supervised learning and feature clustering, which can accurately grade cancer, help to timely detect the disease and formulate treatment plans.

[0219] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting cancer grade, characterized in that: include: Step S1: Collect the patient's cervical squamous cell carcinoma pathological images and their corresponding grade annotation information; Grading categories include: no lesions, mild cervical lesions, moderate to severe cervical lesions, and other atypical representatives; Step S2: preprocessing each collected pathological image; Step S3: Divide each preprocessed pathological image into several image blocks; Step S4: using the image blocks of each pathological image obtained in step S3 to train a self-supervised learning model, wherein the self-supervised learning model is used to divide each image block into a number of image slices, extract image slice features of each image slice, and obtain an image block encoding vector for each image block; Step S5: clustering the image block coding vectors obtained in step S4 using a clustering algorithm, clustering the image block coding vectors of each pathological image into several category regions; Step S6: performing feature sampling on the image block coding vectors of each category area, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of each pathological image; The step S6 comprises: Step S6.1: performing feature sampling on the image block coding vectors of each category region of each pathological image, and selecting representative image block coding vectors in each category region; Step S6.2: Perform feature concatenation on all the image block encoding vectors within each category region after sampling to obtain the regional fusion features of each category region; Step S6.3: performing feature splicing on the regional fusion features of all category regions of each pathological image to obtain the image fusion features of each pathological image; Step S7: Using the image fusion features with category region information of each pathological image obtained in step S6, a cancer grading model is trained. The cancer grading model is based on category region information and can capture subtle changes in pathological images, including cell morphology, structural heterogeneity, and characteristics of the tumor microenvironment. It considers the influence between different category regions and the influence within the category regions, and is used to predict the grade of cervical squamous cell carcinoma. Step S8: Apply the trained model to the pathological image to be analyzed to predict the grade category of cervical squamous cell carcinoma.

2. The cancer grade prediction method according to claim 1, characterized in that: The preprocessing operations in step S2 include: removing image noise, removing image artifacts, removing image background, and data augmentation.

3. The cancer grade prediction method according to claim 1, wherein: The step S8 comprises: Step S8.1: Collect the patient's pathological images to be analyzed; Step S8.2: pre-processing the collected pathological images to be analyzed; Step S8.3: Divide the preprocessed pathological image into a number of image blocks; Step S8.4: Divide the image block of the pathological image to be analyzed obtained in step S8.3 into a number of image slices using the trained self-supervised learning model, extract the image slice features of each image slice, and obtain the image block encoding vector of each image block; Step S8.5: clustering the image block coding vectors obtained in step S8.4 using a clustering algorithm, clustering the image block coding vectors of the pathological image to be analyzed into several category regions; Step S8.6: performing feature sampling on the image block coding vectors of each category region, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of the pathological image to be analyzed; Step S8.7: Based on the image fusion features of the pathological image to be analyzed obtained in step S8.6, the patient's cancer grade is predicted using the trained cancer grade model.

4. A cancer grade prediction device, characterized in that: include: A collection module is used to collect patients' cervical squamous cell carcinoma pathological images and their corresponding graded annotation information; Grading categories include: no lesions, mild cervical lesions, moderate to severe cervical lesions, and other atypical representatives; A preprocessing module, used to preprocess each collected pathological image; A division module, used for dividing each preprocessed pathological image into a number of image blocks; a self-supervised learning model training module, configured to train a self-supervised learning model using the image blocks of each pathological image obtained by the segmentation module, wherein the self-supervised learning model is configured to divide each image block into a plurality of image slices, extract image slice features of each image slice, and obtain an image block encoding vector for each image block; A clustering module is used to cluster the image block coding vectors obtained by the self-supervised learning model training module using a clustering algorithm, and cluster the image block coding vectors of each pathological image into several category regions; A sampling and fusion module is used to perform feature sampling on the image block coding vectors of each category area and perform feature fusion on the sampled image block coding vectors to obtain image fusion features of each pathological image; The sampling fusion module includes: A sampling unit, configured to perform feature sampling on the image block coding vectors of each category region of each pathological image, and select representative image block coding vectors in each category region; The first fusion unit is used to perform feature splicing on all the image block coding vectors in each category area after sampling to obtain the regional fusion features of each category area; The second fusion unit is used to perform feature splicing on the regional fusion features of all category regions of each pathological image to obtain the image fusion features of each pathological image; A cancer grading model training module is used to train a cancer grading model using the image fusion features with category region information for each pathology image obtained by the sampling and fusion module. Based on category region information, the cancer grading model can capture subtle changes in pathology images, including cell morphology, structural heterogeneity, and characteristics of the tumor microenvironment. It considers the influence between different category regions and the influence within category regions to predict the grade of cervical squamous cell carcinoma. The application module is used to apply the trained model to the pathological images to be analyzed and predict the grade category of cervical squamous cell carcinoma.

5. The cancer grade prediction device according to claim 4, characterized in that: The preprocessing operations in the preprocessing module include: removing image noise, removing image artifacts, removing image background, and data augmentation.

6. The cancer grade prediction device according to claim 4, characterized in that: The application module includes: An application collection unit is used to collect the patient's pathological images to be analyzed; An application preprocessing unit is used to preprocess the collected pathological images to be analyzed; Applying a division unit, for dividing the preprocessed pathological image into a plurality of image blocks; an application feature extraction unit, configured to divide the image block of the pathological image to be analyzed obtained by the application division unit into a plurality of image slices using the trained self-supervised learning model, and extract image slice features of each image slice to obtain an image block coding vector for each image block; An application clustering unit is used to cluster the image block coding vectors obtained by the application feature extraction unit using a clustering algorithm, and cluster the image block coding vectors of the pathological image to be analyzed into a plurality of category regions; Applying a sampling and fusion unit to perform feature sampling on the image block coding vectors of each category area, and performing feature fusion on the sampled image block coding vectors to obtain image fusion features of the pathological image to be analyzed; The application grading unit is used to predict the cancer grade of the patient using the trained cancer grading model based on the image fusion features of the pathological image to be analyzed obtained by the application sampling fusion unit.

7. A computing device, characterized in that include: one or more processors; Memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, and the one or more programs include instructions of the cancer grade prediction method according to any one of claims 1 to 3.

8. A storage medium, characterized in that The storage medium stores one or more computer-readable programs, wherein the one or more programs include instructions, and the instructions are suitable for being loaded by the memory and executing the cancer grade prediction method according to any one of claims 1 to 3.

Images