BRAF mutation prediction method and system for papillary thyroid carcinoma

Abstract

The invention discloses a papillary thyroid carcinoma BRAF mutation prediction method and system, and the method comprises the steps: obtaining a full image of PTC, and carrying out the quality control, segmentation, preliminary screening and dyeing standardization processing, and obtaining candidate image blocks; performing cell nucleus segmentation, classification and secondary screening processing on the candidate image blocks to obtain key image blocks; performing cell channel decoupling on the key image blocks, and respectively extracting cell-level pathological omics manual features of different channels for aggregation processing to obtain patient-level feature vectors; and carrying out feature selection, splicing fusion and cross validation on the patient feature vectors to construct an optimal model, and carrying out reasoning and voting to obtain a BRAF mutation prediction result. By using the method, potential important feature information carried by different cells can be extracted, so that the accuracy and interpretability of BRAF mutation prediction of PTC are improved. The PTC BRAF mutation prediction method and system can be widely applied to the technical field of gene mutation prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of gene mutation prediction, and in particular to a method and system for predicting BRAF mutation in papillary thyroid carcinoma. Background Art

[0002] Papillary thyroid carcinoma (PTC) is the most common type of thyroid cancer. BRAF gene mutation, especially BRAF-V600E mutation, is one of the most common molecular changes in PTC. Currently, the main methods for predicting BRAF mutation in PTC include PCR (polymerase chain reaction) gene detection analysis and pathological analysis based on IHC (immunohistochemistry) and FNAC (fine needle aspiration cytology biopsy). Although PCR gene detection analysis is the "gold standard" for detecting BRAF mutation, it is costly, time-consuming, and the detection facilities and technology are relatively complex, which is not conducive to early screening. IHC protein expression analysis plays an important role in BRAF mutation screening and diagnosis, but it requires postoperative slices, resulting in delayed results and unable to guide the formulation of preoperative treatment strategies in a timely manner. FNAC is the most economical and commonly used pathological diagnostic method for early screening of PTC, but the analysis of fine needle aspiration cytology smears for PTC still has some shortcomings. For example, the full-frame image obtained by digital scanning of cytology smears usually has a very high resolution and large size, and the size of a single image may reach hundreds of megabytes or even thousands of gigabytes. Processing such a huge amount of data requires high computing resources and storage space. Manual analysis requires viewing and evaluating the entire high-resolution image one by one, which is very time-consuming and prone to missed detection. In the cytological smears of early screening, cancer cells may be very sparse and scattered. It takes a high degree of concentration and rich experience to locate these sparse cancer cells with the naked eye, which can easily lead to fatigue and missed diagnosis. The morphology of sparse cancer cells may be very similar to benign cells, which increases the difficulty of identification and the risk of misdiagnosis. Long-term naked eye observation and manual analysis will cause visual fatigue and decreased attention of pathologists, affecting the quality of diagnosis. Manual analysis is inefficient and difficult to cope with the processing needs of large-scale pathological image data, affecting the timeliness of diagnosis. Detecting subtle cytological changes and abnormalities requires a high level of technology and expertise. Pathologists may be affected by personal experience and subjective factors when analyzing smears, which affects the repeatability and consistency of the results. In addition, most of the existing PTC auxiliary diagnosis models are modeled based on deep features. Although they can capture complex patterns at the pixel level, they lack intuitive interpretability. It is difficult for doctors to understand the decision-making process and logic of the model and build trust, which hinders the widespread deployment and application of the model. Summary of the invention

[0003] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for predicting BRAF mutation in papillary thyroid carcinoma, which can extract potentially important characteristic information carried by different cells from a pathological perspective, thereby improving the accuracy and interpretability of the prediction results of BRAF mutation in papillary thyroid carcinoma.

[0004] The first technical solution adopted by the present invention is: a method for predicting BRAF mutation in papillary thyroid carcinoma, comprising the following steps:

[0005] Obtain full-frame fine needle aspiration cytology images of papillary thyroid carcinoma and perform quality control, segmentation, initial screening and staining standardization to obtain standardized candidate image blocks;

[0006] Perform cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks;

[0007] The key image blocks are decoupled by cell channels, and the cell-level pathological omics manual features of different channels are extracted and aggregated to obtain the patient-level feature vector;

[0008] The patient-level feature vectors are divided into a training set and a test set. Feature selection and splicing and fusion processing are performed based on the training set to train the model. The important features of each channel in the training set are selected, and the feature matrix of the test set is pruned and spliced ​​and fused to build a single-channel optimal model and a multi-channel optimal fusion model, and determine the important feature set of a single channel and the fusion feature set;

[0009] The important feature sets of each single channel are input into the corresponding single-channel optimal model for reasoning, and the fusion feature sets are input into the multi-channel optimal fusion model for reasoning. All model results are voted according to the preset weights to obtain the BRAF mutation prediction results.

[0010] Furthermore, the step of obtaining a full-frame fine needle aspiration cytology image of papillary thyroid carcinoma and performing quality control, segmentation, primary screening and staining standardization to obtain a standardized candidate image block specifically includes:

[0011] Obtain Regis-stained fine needle aspiration cytology smears of papillary thyroid carcinoma and perform digital scanning to obtain full-frame pathological images;

[0012] The contaminated area and background area of ​​the full-frame image are filtered through the open source model HistoQC to obtain the computable area of ​​the filtered full-frame image.

[0013] The computable area is segmented, and preliminary screening is performed by calculating the tissue proportion, information entropy and contrast of the segmented image blocks to obtain candidate image blocks;

[0014] The candidate image blocks are dyed and standardized by HistomicsTK to make the dyeing styles of different image blocks uniform and obtain the standardized candidate image blocks.

[0015] Furthermore, the step of performing cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks specifically includes:

[0016] The red blood cells and blank areas in the candidate image blocks are divided into background areas through the open source model Ilastik, and the cell nuclei therein are segmented to obtain the cell nucleus segmentation results of the candidate image blocks;

[0017] The cell nucleus segmentation results in the candidate image blocks are classified using the open source model Ilastik to obtain the cell nucleus classification results;

[0018] Counting processing is performed according to the cell nucleus classification results, and candidate image blocks corresponding to tumor cell nucleus count rankings greater than a preset threshold are selected as key image blocks.

[0019] Furthermore, the step of performing cell channel decoupling on the key image blocks, extracting cell-level pathological omics manual features of different channels for aggregation processing, and obtaining a patient-level feature vector specifically includes:

[0020] According to the results of cell nucleus segmentation and classification, the key image blocks are decoupled into cell channels, and 5 cell channel masks are obtained from each key image block;

[0021] Extract the pathological omics manual features of the corresponding types of cell nuclei of all key image blocks under the masks of the five cell channels to form the cell-level feature matrix of the five separate cell channels;

[0022] The mean eigenvector and variance eigenvector corresponding to the cell-level feature matrix of different cell channels are obtained respectively, and vector concatenation processing is performed to realize feature aggregation, so as to obtain a patient-level feature vector with several cell channels.

[0023] Furthermore, the cell-level pathological omics manual features include cell nucleus shape features, cell nucleus texture features, cell co-occurrence features and cell density features, wherein:

[0024] The cell nucleus shape feature is obtained by calculating the basic morphology of the cell nucleus, and the cell nucleus shape feature is used to quantify the complexity of the cell nucleus contour and regional attributes;

[0025] The cell nucleus texture feature is obtained by calculating the first-order statistics of pixel intensity in the cell nucleus region and the second-order statistics based on the gray-level co-occurrence matrix, and the cell nucleus texture feature is used to quantify the complexity of the cell nucleus texture;

[0026] The cell co-occurrence feature is obtained by calculating the probability that the cell nucleus is predicted to be 5 different cell nucleus types, and the cell co-occurrence feature is used to quantify the degree of cell nucleus heteromorphism or the transition state between different cells;

[0027] The cell density feature is obtained by calculating the number of cells in all key image blocks in the full image and dividing it by the global tissue region area. The cell density feature is used to quantify the cellular components of the tumor microenvironment.

[0028] Furthermore, the patient-level feature vector is divided into a training set and a test set, feature selection and splicing and fusion processing are performed based on the training set, model training is performed, important features of each channel in the training set are selected, pruning and splicing and fusion processing are performed on the feature matrix of the test set, a single-channel optimal model and a multi-channel optimal fusion model are constructed, and the step of determining the important feature set of a single channel and the fusion feature set specifically includes:

[0029] Based on the training phase, the patient-level feature vectors having several cell channels are respectively formed into a patient-level feature matrix for each cell channel;

[0030] The Lasso regression feature selection algorithm is used to select features from the patient-level feature matrix of each cell channel to obtain several important discriminative features in each channel. The corresponding patient-level important feature submatrix of each channel is input into the corresponding single-channel model for training;

[0031] The important features selected from all channels are concatenated and fused, and after feature selection, several more discriminative fusion features are obtained. The corresponding fusion feature matrix is ​​input into the multi-channel fusion model for training;

[0032] Based on the test phase, the patient-level feature vectors of several cell channels are respectively composed into the patient-level feature matrix of each cell channel, and pruned according to the important features of each channel selected in the training phase, and the corresponding important feature sub-matrices are input into the corresponding single-channel model for performance evaluation to build the single-channel optimal model;

[0033] The patient-level feature matrices of all channels are pruned and concatenated according to the fusion features selected in the training phase. The obtained fusion feature matrix is ​​input into the multi-channel fusion model for performance evaluation to build the multi-channel optimal fusion model.

[0034] The hyperparameters of each single-channel model and fusion model are adjusted through ten-fold cross validation, and the important features of different channels and fusion features are scored. The optimal model of each single channel and the optimal fusion model are saved, and the important feature sets and fusion feature sets of each single channel with a score ranking higher than the preset ranking are determined.

[0035] Furthermore, the fusion feature set includes texture features, shape features and cell co-occurrence features, wherein:

[0036] The texture features include the range of variance-aggregated tumor cell nucleus correlation, the kurtosis of variance-aggregated tumor cell nucleus pixel intensity, the skewness of variance-aggregated tumor cell nucleus pixel intensity, and the range of variance-aggregated average contrast of other types of cell nuclei;

[0037] The shape characteristics include the standard deviation of the variance-aggregated area of ​​other types of cell nuclei and the average aggregated center-to-center distance of the adherent normal cell nuclei;

[0038] The cell co-occurrence features include the average probability that aggregated cohesive normal cell nuclei are predicted to be other types of cell nuclei.

[0039] Furthermore, the step of inputting each single-channel important feature set into the corresponding single-channel optimal model for reasoning and inputting the fusion feature set into the multi-channel optimal fusion model for reasoning, and voting all model results according to preset weights to obtain the BRAF mutation prediction result specifically includes:

[0040] Based on the deployment phase, the patient-level feature vectors of each channel of the new patient are obtained, and pruned according to the important feature set of the single channel to obtain the corresponding important feature sub-vectors and input them into the single channel optimal model for reasoning to obtain the first BRAF mutation prediction probability;

[0041] The patient-level feature vectors of all channels are pruned and concatenated according to the fusion feature set determined in the modeling stage, and the fusion feature vector is input into the multi-channel optimal fusion model for reasoning to obtain the second BRAF mutation prediction probability;

[0042] According to the first BRAF mutation prediction probability and the second BRAF mutation prediction probability, weights are assigned and voted to obtain the BRAF mutation prediction result.

[0043] The second technical solution adopted by the present invention is: a system for predicting BRAF mutation in papillary thyroid carcinoma, comprising:

[0044] The first module is used to obtain the full-frame fine needle aspiration cytology image of papillary thyroid carcinoma and perform quality control, segmentation, initial screening and staining standardization to obtain standardized candidate image blocks;

[0045] The second module is used to perform cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks;

[0046] The third module is used to decouple the key image blocks into cell channels and extract the cell-level pathological omics manual features of different channels for aggregation processing to obtain the patient-level feature vector;

[0047] The fourth module is used to divide the patient-level feature vector into a training set and a test set, perform feature selection and splicing and fusion processing based on the training set, perform model training, select important features of each channel in the training set, perform pruning and splicing and fusion processing on the feature matrix of the test set, build a single-channel optimal model and a multi-channel optimal fusion model, and determine the important feature set of a single channel and the fusion feature set;

[0048] The fifth module is used to input the important feature sets of each single channel into the corresponding single-channel optimal model for reasoning and input the fusion feature set into the multi-channel optimal fusion model for reasoning, and vote all model results according to the preset weights to obtain the BRAF mutation prediction results.

[0049] The beneficial effects of the method and system of the present invention are as follows: the present invention obtains the full-frame fine needle aspiration cytology image of thyroid papillary carcinoma and performs quality control, segmentation, primary screening and staining standardization to obtain candidate image blocks, and can filter out the blurred, handwriting, stains, over-darkly stained and other polluted areas and blank background areas in the global range of the full-frame image to obtain candidate image blocks, so as to ensure that the image blocks finally input into the model have a certain number of cells; in addition, image blocks from different WSIs may have batch effects caused by operations such as film preparation, staining, and imaging. The candidate image blocks are stained and standardized by HistomicsTK, so that the staining styles of different image blocks are unified, which can improve the robustness of the model; the candidate image blocks are further segmented, classified and secondary screened to obtain key image blocks. The key image blocks selected according to the tumor cell count are in line with the doctor's reading habits and can avoid tedious searching and searching in the full-frame image, greatly improving work efficiency; the key image blocks are cell channel selected according to the cell nucleus classification results. Decoupling can avoid the error of treating the connected domains of multiple adherent cell nuclei as one cell nucleus when extracting features, and can prevent the potential important information carried by different types of cell nuclei from being covered or interfering with each other during feature aggregation. Then, pathological genomics manual features are extracted in different cell channels respectively to obtain important information that quantitatively describes different types of cells in key image blocks, and aggregation processing is performed to obtain patient-level feature vectors of each channel. This process can reduce the loss of local information to a certain extent when performed in different cell channels; feature selection of patient-level features of each channel can reduce the risk of overfitting, and the selected important features are used for single-channel model training, and can also be used as an explanation basis for model reasoning for doctors' reference; the selected important features are spliced ​​and fused, and further feature selection is performed, and the obtained fusion features are used for training multi-channel fusion models, which can improve the accuracy of BRAF mutation prediction in papillary thyroid carcinoma; the feature vectors of the patients to be tested are pruned, spliced, model-reasoned and voted in turn to obtain the prediction results of BRAF mutation in papillary thyroid carcinoma. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of the steps of a method for predicting BRAF mutation in papillary thyroid carcinoma of the present invention;

[0051] Figure 2 It is a structural block diagram of a system for predicting BRAF mutation in papillary thyroid carcinoma according to the present invention;

[0052] Figure 3 is an implementation block diagram of BRAF mutation prediction provided by an embodiment of the present invention;

[0053] Figure 4 It is a schematic diagram of the deployment of a model for BRAF mutation prediction provided by an embodiment of the present invention;

[0054] Figure 5 It is a schematic diagram of model training and testing for BRAF mutation prediction provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The step numbers in the following embodiments are only provided for the convenience of explanation and description, and the order between the steps is not limited in any way. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0056] Reference Figure 1 and Figure 3 The present invention provides a method for predicting BRAF mutation in papillary thyroid carcinoma, the method comprising the following steps:

[0057] S100, obtaining a full-frame fine needle aspiration cytology image of papillary thyroid carcinoma and performing quality control, segmentation, primary screening and staining standardization processing to obtain a standardized candidate image block;

[0058] Specifically, a Regis-stained fine needle aspiration cytology smear of papillary thyroid carcinoma was obtained and digitally scanned to obtain a full-frame image; the background area and contaminated area of ​​the full-frame image were filtered using the open source model HistoQC to obtain a computable area of ​​the filtered full-frame image; the computable area was segmented, and a preliminary screening was performed by calculating the tissue proportion, information entropy and contrast of the segmented image blocks to obtain candidate image blocks; the candidate image blocks were stained and standardized using HistomicsTK to unify the staining styles of different image blocks.

[0059] In this embodiment, compared with H&E stained sections, the quality of Regis stained cytological smears varies, and large areas of invalid areas with too deep staining are prone to occur during the film making process, and the imaging effect is poor. Therefore, it is indispensable to perform quality control on the WSI of cytological smears and screen out the computable areas. This system first filters the areas of blur, handwriting, stains, and contaminated areas such as too deep staining and blank backgrounds in the global range of WSI through the open source model HistoQC, and divides the computable areas obtained after quality control into image blocks for input model and subsequent processing. These image blocks representing local areas need to be preliminarily screened by calculating their tissue proportion, information entropy, and contrast to obtain candidate image blocks to ensure that the image blocks finally input to the model have a certain number of cells. In addition, image blocks from different WSIs may have batch effects caused by operations such as film making, staining, and imaging. The candidate image blocks are stained and standardized by HistomicsTK, so that the staining styles of different image blocks are unified, which can improve the robustness of the model.

[0060] S200, performing cell nucleus segmentation, classification and secondary screening processing on the standardized candidate image blocks to obtain key image blocks;

[0061] Specifically, the red blood cells and blank areas in the candidate image blocks are divided into background areas through the open source model Ilastik, and the cell nuclei therein are segmented to obtain the cell nucleus segmentation results of the candidate image blocks; the cell nucleus segmentation results in the candidate image blocks are classified through the open source model Ilastik, and the classification strategy is five categories: tumor cell nuclei, adhesion tumor cell nuclei, normal follicular cell nuclei, adhesion normal follicular cell nuclei and other types of cell nuclei, to obtain cell nucleus classification results; the cell nucleus classification results of the candidate image blocks are counted, and the candidate image blocks corresponding to the tumor cell nucleus count ranking greater than the preset threshold are selected as key image blocks.

[0062] In this embodiment, first, the open source model Ilastik is used to divide the cell nuclei into five categories, namely tumor_nuclei (tumor cell nuclei), merge_tumor_nuclei (adhesion tumor cell nuclei), normal_nuclei (normal follicular cell nuclei), merge_normal_nuclei (adhesion normal follicular cell nuclei), others (other types of cell nuclei, mainly neutrophil nuclei and fibrous impurities), and the red blood cells and blank areas are divided into background areas. On the one hand, this classification strategy conforms to the doctor's reading habits, that is, it is necessary to focus on tumor cells for targeted analysis, and on the other hand, it is based on engineering considerations, that is, cytological smears are more likely to have multiple cell nuclei in space due to the way they are made. The phenomenon of overlapping or merging (merge) needs to be distinguished from independent non-adhesive cell nuclei to avoid the error of treating the connected domain of multiple adhesion cell nuclei as one cell nucleus when extracting features.

[0063] Secondly, the candidate image blocks that were initially screened were sorted according to the tumor cell count, and the first 128 image blocks were selected as key image blocks (Key Patches) for subsequent further processing, modeling and analysis. At the same time, they were provided to doctors for observation, avoiding tedious searching in the full image, greatly improving work efficiency, which is also one of the important manifestations of model interpretability.

[0064] S300, decoupling the key image blocks into cell channels, extracting the cell-level pathological omics manual features of different channels for aggregation processing, and obtaining the patient-level feature vector;

[0065] Specifically, the key image blocks were cell channel decoupled according to the cell nucleus classification results, and 5 cell channel masks were obtained from each key image block; the pathological omics manual features of the corresponding types of cell nuclei of all key image blocks under the 5 cell channel masks were extracted respectively, and the cell-level feature matrices of 5 separate cell channels were obtained; the mean eigenvectors and variance eigenvectors corresponding to the cell-level feature matrices of different cell channels were obtained respectively, and vector splicing processing was performed to realize feature aggregation and obtain the patient-level feature vectors of each cell channel.

[0066] In this embodiment, first, the key image blocks are innovatively decoupled from the cell channels according to the cell nucleus classification results to avoid the error of treating the connected domains of multiple adhesion cell nuclei as one cell nucleus when extracting features, and to prevent the potential important information carried by different types of cell nuclei from being covered or interfering with each other during feature aggregation; the manual feature extractor is used to extract the cell-level pathology manual features in different cell channels to obtain important information that quantitatively describes different types of cell nuclei in the key image blocks; since the clinical information label, that is, whether the patient has a BRAF mutation, is obtained by PCR detection, the label belongs to the patient level, so it is necessary to aggregate the local cell-level feature vector into a global patient-level feature vector to match the label, and the cell-level feature matrix composed of the cell-level feature vectors of different channels is calculated by respectively calculating the mean feature vector and variance feature vector of each cell-level feature matrix and splicing them, so as to realize the aggregation of the cell-level feature matrix into a patient-level feature vector that can represent the entire WSI, and this process is carried out in different cell channels to reduce the loss of local information to a certain extent. This embodiment uses a relatively simple non-parametric aggregation method, namely, mean variance aggregation, instead of using a multi-instance learning aggregation method based on deep learning, which is the result of a trade-off between model complexity and interpretability.

[0067] Furthermore, it should be noted that the manual features of cell-level pathological omics include nuclear shape features, nuclear texture features, cell co-occurrence features and cell density features. Among them, the nuclear shape features are calculated by calculating the basic morphological indicators of the nuclear area, perimeter, eccentricity, smoothness, fractal dimension, Fourier descriptor, reverse moment, axial length ratio, roundness, polarity, etc., with a total of 71 dimensions, which are used to quantify the complexity of the nuclear contour and regional attributes; the nuclear texture features are obtained by calculating the first-order statistics of the pixel intensity in the nuclear region and the second-order statistics based on the gray-level co-occurrence matrix (GLCM), such as the intensity mean, intensity variance, mutual information, correlation, entropy, energy and other indicators, with a total of 73 dimensions, which are used to quantify the complexity of the nuclear texture; the cell co-occurrence features are calculated by predicting the nucleus to be 5 The probability of different cell nuclear types is obtained, with a total of 5 dimensions, which is used to quantify the degree of nuclear heteromorphism or the transition state between different cells. When the cell co-occurrence feature is aggregated into a patient-level feature in a certain cell channel, it indicates the probability that the cell type corresponding to the cell channel will co-appear with different cell types in the dense area of ​​tumor cells, so it is called the cell co-occurrence feature; the cell density feature is obtained by calculating the number of cells in all key image blocks in the full image and dividing it by the global tissue area, with a total of 5 dimensions, which is used to quantify the cellular components of the tumor microenvironment; finally, each cell corresponds to a 154-dimensional cell-level pathology manual feature vector.

[0068] S400, dividing the patient-level feature vector into a training set and a test set, performing feature selection and splicing and fusion processing based on the training set, performing model training, selecting important features of each channel in the training set, performing pruning and splicing and fusion processing on the feature matrix of the test set, constructing a single-channel optimal model and a multi-channel optimal fusion model, and determining a single-channel important feature set and a fusion feature set;

[0069] Specifically, in the training phase: the patient-level feature vectors of the five cell channels of the training set containing a certain number of negative and positive patients are obtained through the above steps, and the patient-level feature matrices of the five cell channels are respectively formed; feature selection algorithms such as Lasso regression are used to select features from the patient-level feature matrices of each channel, and several (7-dimensional) important features with discriminativeness are obtained in each channel, and the corresponding patient-level important feature submatrices of each channel are input into the corresponding single-channel model for training; the important features selected from all channels are spliced ​​and fused, and after further feature selection, several (7-dimensional) fusion features with more discriminativeness are obtained, and the corresponding fusion feature matrices are input into the multi-channel fusion model for training. In the testing phase: the patient-level feature vectors of the five cell channels of the test set containing a certain number of patients are obtained through the above steps, and the patient-level feature matrices of the five cell channels are respectively formed, and the important features of each channel selected from the training phase are pruned, and the corresponding important feature submatrices are input into the corresponding single-channel model for performance evaluation; the patient-level feature matrices of all channels are pruned and spliced ​​according to the fusion features selected from the training phase, and the obtained fusion feature matrices are input into the multi-channel fusion model for performance evaluation. The hyperparameters of each single-channel model and fusion model are adjusted through ten-fold cross validation, and the important features of different channels and fusion features are scored. The optimal model of each single channel and the optimal fusion model are saved, and the top 7 important feature sets and fusion feature sets (Top7 Fusion FeatureSet) of each single channel are determined.

[0070] Feature selection of patient-level feature vectors of each channel can reduce the risk of overfitting. The selected important features are used for single-channel model training and serve as an explanation basis for model reasoning for doctors' reference. The selected important features are spliced ​​and fused for further feature selection, and the obtained fusion features are used for training multi-channel fusion models, which can improve the accuracy of BRAF mutation prediction in papillary thyroid carcinoma.

[0071] Furthermore, it should be noted that the fusion feature set includes texture features, shape features and cell co-occurrence features, wherein the texture features include the range of variance-aggregated tumor cell nucleus correlation, the kurtosis of variance-aggregated tumor cell nucleus pixel intensity, the skewness of variance-aggregated tumor cell nucleus pixel intensity and the range of variance-aggregated average contrast of other types of cell nuclei; the shape features include the standard deviation of the area of ​​other types of cell nuclei aggregated by variance and the inner axis distance of the average aggregated adhesion normal cell nuclei; and the cell co-occurrence features include the probability that the average aggregated adhesion normal cell nuclei are predicted to be other types of cell nuclei.

[0072] Specifically, Top7_Fusion_Feature_1 is the range of tumor cell nucleus correlation of variance aggregation, and the calculation expression is:

[0073]

[0074] It belongs to the texture feature, indicating the range of differences in the regularity of the pixel value distribution of different tumor cell nuclei in the Key Patches area. A high correlation means that there are more repeated textures in the tumor cell nuclei. Among them, the range of the regularity of the pixel value distribution of different tumor cell nuclei in the Key Patches area is:

[0075]

[0076] In the above formula, K is the number of Key Patches, C k is the number of tumor cells in the current Key Patch, p kc (i, j) is the (i, j)th element in the standardized GLCM of the cth tumor cell nucleus ROI in the current Key Patch, μ i ,μ j is the mean of the marginal distribution of the GLCM, σ i ,σ j is the standard deviation of the GLCM edge distribution, i,j represents the gray level, that is, the row and column index of the GLCM.

[0077] Top7_Fusion_Feature_2 is the kurtosis of the tumor cell nuclear pixel intensity after variance aggregation, and the calculation expression is:

[0078]

[0079] It belongs to the texture feature, which indicates the difference in the sharpness of the pixel value distribution of different tumor cell nuclei in the Key Patches area. The high peak means that there are more edge patterns in the cell nucleus. Among them, the sharpness of the pixel value distribution of different tumor cell nuclei is:

[0080]

[0081] In the above formula, N is the number of all tumor cell nuclei in the Key Patches area of ​​the entire WSI, M n is the number of pixels in the nth tumor cell nucleus ROI, X(m) represents the intensity of the mth pixel, M n The average intensity of pixels.

[0082] Top7_Fusion_Feature_3 is the skewness of the tumor cell nuclear pixel intensity of variance aggregation, and the calculation expression is:

[0083]

[0084] It belongs to the texture feature, which indicates the difference in the asymmetric distribution of pixel values ​​of different tumor cell nuclei in the Key Patches area. High skewness means that there are more highlights or black spots in the cell nucleus. Among them, the asymmetric distribution of pixel values ​​of different tumor cell nuclei is:

[0085]

[0086] In the above formula, N is the number of all tumor cell nuclei in the Key Patches area of ​​the entire WSI, M n is the number of pixels in the nth tumor cell nucleus ROI, X(m) represents the intensity of the mth pixel, M n The average intensity of pixels.

[0087] Top7_Fusion_Feature_4 is the area of ​​other types of cell nuclei for variance aggregation, and the calculation expression is:

[0088]

[0089] It belongs to the shape feature, indicating the difference in the area size of different other cell nuclei in the Key Patches region, where N is the number of all other types of cells in the Key Patches region of the entire WSI, area(n) = M n , represents the number of pixels in the nth other type of cell nucleus ROI.

[0090] Top7_Fusion_Feature_5 is the probability that the average aggregated adhesion normal cell nucleus is predicted to be other types of cell nuclei. The calculation expression is:

[0091]

[0092] It is a cell co-occurrence feature, indicating the similarity between the adhesion nuclei of each cluster of normal cells and other types of cell nuclei in the Key Patches area where tumor cells are more enriched. It also quantifies the probability that normal cell clusters with adhesion nuclei and other types of cells appear together in the tumor cell-dense area to a certain extent, where N is the number of all adhesion normal cell nuclei in the Key Patches area of ​​the entire WSI.

[0093] Top7_Fusion_Feature_6 is the standard deviation of the average polymerized adhesion normal cell nuclear inner axis distance, and the calculation expression is:

[0094]

[0095] It belongs to the shape feature, which indicates the degree of distortion of the shape of the adhesion cell nucleus connected domain of each group of normal cells in the key area corresponding to the Key Patches. Among them, the degree of distortion of the shape of the adhesion cell nucleus connected domain of each group of normal cells is:

[0096]

[0097] In the above formula, N is the number of all normal cell nuclei adhered to the Key Patches area of ​​the entire WSI, V n is the number of contour vertices of the connected domain of the nth adhesion normal cell nucleus, d(v) represents the distance from the vth vertex to the centroid, V n The average distance from the vertices to the centroid.

[0098] Top7_Fusion_Feature_7 is the range of the average contrast of other types of cell nuclei aggregated by variance, and the calculation expression is:

[0099]

[0100] It belongs to the texture feature, indicating the difference in the range of the difference between the adjacent pixel values ​​of different types of other types of cell nuclei in the Key Patches area. High contrast means that there are significant grayscale differences in other types of cell nuclei, which usually means that the texture in the cell nucleus ROI is rough, with obvious edges and lines. Among them, the texture in the cell nucleus ROI is rough:

[0101]

[0102] In the above formula, K is the number of Key Patches, C k is the number of other types of cells in the current Key Patch, p kc(i, j) is the (i, j)th element in the standardized GLCM of the cth other type of cell nucleus ROI in the current Key Patch, and i, j represents the gray level, that is, the row and column index of the GLCM.

[0103] S500, inputting each single-channel important feature set into the corresponding single-channel optimal model for reasoning, and inputting the fusion feature set into the multi-channel optimal fusion model for reasoning, and voting all model results according to preset weights to obtain BRAF mutation prediction results.

[0104] Specifically, in the deployment phase, the patient-level feature vectors of each channel of the new patient are obtained through the above steps, and pruned according to the important feature sets of each channel determined in the modeling phase, and the corresponding important feature sub-vectors are input into the corresponding single-channel optimal model for inference to obtain the BRAF mutation prediction probability; the patient-level feature vectors of all channels are pruned and concatenated according to the fusion feature set determined in the modeling phase, and the fusion feature vectors are input into the optimal multi-channel fusion model for inference to obtain the BRAF mutation prediction probability; the prediction probabilities of different single-channel models and fusion models are voted with certain weights to obtain the final prediction result.

[0105] In summary, if Figure 4 and Figure 5 As shown in FIG. 1 , the main processing flow of the embodiment of the present invention is to obtain the full-frame fine needle aspiration cytology image of thyroid papillary carcinoma and perform quality control, segmentation, primary screening and staining standardization to obtain candidate image blocks at high magnification; perform cell nucleus segmentation, classification, and secondary screening based on tumor cell count on the candidate image blocks to obtain key image blocks (Key Patches); perform cell channel decoupling, manual feature extraction and feature aggregation for each key image block to obtain patient-level feature vectors for different cell channels; in the modeling stage, perform feature selection and splicing fusion processing on the feature matrix composed of the patient-level feature vectors of each channel of the patients in the training set for model training; perform pruning and splicing fusion processing on the feature matrix composed of the patient-level feature vectors of the patients in the test set according to the important features of each channel selected from the training set, and input the corresponding model for performance evaluation; adjust the hyperparameters of different models through cross-validation, save the optimal model of each single channel and the optimal multi-channel fusion model, and determine the important feature set and fusion feature set of each single channel; in the deployment stage, perform pruning and splicing fusion processing on the patient-level feature vectors of new patients according to the feature set determined in the modeling stage, input the corresponding optimal model for reasoning and voting, and obtain the BRAF mutation prediction results. Doctors can make a more comprehensive, detailed and rapid judgment on the BRAF mutation status of patients' papillary thyroid carcinoma by referring to the prediction results and final voting results of different channel models, combined with the Key Patches area of ​​interest of the observation model, so as to formulate a more reasonable treatment plan.

[0106] Therefore, the embodiments of the present invention have the following advantages over the prior art:

[0107] 1) In terms of interpretability, the model decision-making process is more transparent, the features used in modeling can be traced back to actual cell entities, and the model's focus on tumor cell-enriched areas is also in line with doctors' reading habits. The important feature set determined by the model, the key image blocks that the model focuses on, and their positions in the full image are output together with the model reasoning results, which can facilitate doctors to check the model reasoning results and greatly reduce the burden on doctors to read films. The final model prediction is well interpretable.

[0108] 2) In terms of model optimization, the embodiments of the present invention innovatively introduce cell channel decoupling in the feature extraction process to avoid the error of regarding the connected domain of multiple adhesion nuclei as one cell nucleus, prevent the potentially important information carried by different types of cell nuclei from being covered or interfering with each other during feature aggregation, and reduce the loss of local information to a certain extent, thereby obtaining more robust modeling features; on the other hand, it innovatively introduces cell co-occurrence features to quantify the excessive relationship or co-presentation relationship of different cells, significantly improving the model performance.

[0109] Finally, in order to verify the performance of the BRAF mutation prediction system for papillary thyroid carcinoma of the present invention, cross-validation was performed on the papillary thyroid carcinoma dataset using different single-channel models and multi-channel fusion models, and compared with the current SOTA deep learning model MuCRL in the same field, as shown in Table 1.

[0110] Table 1 Comparison of experimental model performance

[0111] Channel ACC AUC F1_score Precision Recall Tumor 0.686±0.091 0.757±0.096 0.700±0.086 0.705±0.109 0.712±0.122 MG_tumor 0.638±0.121 0.710±0.117 0.593±0.122 0.745±0.193 0.531±0.161 Normal 0.559±0.083 0.597±0.084 0.593±0.096 0.582±0.116 0.636±0.153 MG_normal 0.653±0.088 0.719±0.087 0.684±0.082 0.674±0.124 0.721±0.115 Others 0.712±0.074 0.779±0.074 0.732±0.086 0.706±0.084 0.778±0.135 Fusion 0.754±0.083 0.819±0.078 0.782±0.078 0.743±0.119 0.843±0.087 MuCRL 0.559±0.065 0.610±0.077 0.667±0.038 0.560±0.0554 0.842±0.125

[0112] Among them, Channel represents different channel models, Tumor is a single-channel model of tumor cell nuclei, MG_tumor is a single-channel model of adhesion tumor cell nuclei, Normal is a single-channel model of normal cell nuclei, MG_normal is a single-channel model of adhesion normal cell nuclei, Others is a single-channel model of other types of cell nuclei, Fusion is a multi-channel fusion model, and MuCRL is a SOTA model; ACC represents accuracy, AUC represents the area under the receiver operating characteristic curve (ROC), Precision represents precision, Recall represents recall, and F1_score represents the harmonic mean of precision and recall, all of which are commonly used indicators for evaluating model performance; further, in order to verify the effect of the embodiment of the present invention on improving the traditional pathological genomics model, an ablation experiment was performed on the same papillary thyroid carcinoma dataset, as shown in Table 2.

[0113] Table 2 Ablation experiment model performance

[0114] CCD CCC Acc Auc F1_score Precision Recall √ × 0.729±0.069 0.815±0.076 0.759±0.068 0.725±0.110 0.816±0.100 × √ 0.592±0.101 0.646±0.109 0.636±0.098 0.601±0.109 0.701±0.138 √ √ 0.754±0.083 0.819±0.078 0.782±0.078 0.743±0.119 0.843±0.087

[0115] Among them, CCD stands for cell channel decoupling, and CCC stands for cell co-occurrence.

[0116] Reference Figure 2 , a BRAF mutation prediction system for papillary thyroid carcinoma, comprising:

[0117] The first module 201 is used to obtain a full-frame fine needle aspiration cytology image of papillary thyroid carcinoma and perform quality control, segmentation, primary screening and staining standardization processing to obtain a standardized candidate image block;

[0118] The second module 202 is used to perform cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks;

[0119] The third module 203 is used to perform cell channel decoupling on the key image blocks, and extract the cell-level pathological omics manual features of different channels for aggregation processing to obtain the patient-level feature vector;

[0120] The fourth module 204 is used to divide the patient-level feature vector into a training set and a test set, perform feature selection and splicing and fusion processing based on the training set, perform model training, select important features of each channel in the training set, perform pruning and splicing and fusion processing on the feature matrix of the test set, build a single-channel optimal model and a multi-channel optimal fusion model, and determine a single-channel important feature set and a fusion feature set;

[0121] The fifth module 205 is used to input each single-channel important feature set into the corresponding single-channel optimal model for reasoning and input the fusion feature set into the multi-channel optimal fusion model for reasoning, and vote all model results according to preset weights to obtain BRAF mutation prediction results.

[0122] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0123] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A method for predicting BRAF mutation in papillary thyroid carcinoma, characterized in that: The following steps are involved: Obtain full-frame fine needle aspiration cytology images of papillary thyroid carcinoma and perform quality control, segmentation, initial screening and staining standardization to obtain standardized candidate image blocks; Perform cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks; Decouple the key image blocks by cell channels, extract the cell-level pathological omics manual features of different channels, and aggregate them to obtain the patient-level feature vector; The patient-level feature vectors are divided into a training set and a test set. Feature selection and splicing and fusion processing are performed based on the training set to train the model. The important features of each channel in the training set are selected, and the feature matrix of the test set is pruned and spliced ​​and fused to build a single-channel optimal model and a multi-channel optimal fusion model, and determine the important feature set of a single channel and the fusion feature set; The important feature sets of each single channel are input into the corresponding single-channel optimal model for reasoning, and the fusion feature sets are input into the multi-channel optimal fusion model for reasoning. All model results are voted according to the preset weights to obtain the BRAF mutation prediction results.

2. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 1, characterized in that: The step of obtaining a full-frame fine needle aspiration cytology image of papillary thyroid carcinoma and performing quality control, segmentation, primary screening and staining standardization to obtain a standardized candidate image block specifically includes: Obtain Regis-stained fine needle aspiration cytology smears of papillary thyroid carcinoma and perform digital scanning to obtain full-frame pathological images; The contaminated area and background area of ​​the full-frame image are filtered using the open source model HistoQC to obtain the computable area of ​​the filtered full-frame image. The computable area is segmented, and preliminary screening is performed by calculating the tissue proportion, information entropy and contrast of the segmented image blocks to obtain candidate image blocks; The candidate image blocks are dyed and standardized by HistomicsTK to make the dyeing styles of different image blocks uniform and obtain the standardized candidate image blocks.

3. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 2, characterized in that: The step of performing cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks specifically includes: The red blood cells and blank areas in the candidate image blocks are divided into background areas through the open source model Ilastik, and the cell nuclei therein are segmented to obtain the cell nucleus segmentation results of the candidate image blocks; The cell nucleus segmentation results in the candidate image blocks are classified using the open source model Ilastik to obtain the cell nucleus classification results; Counting processing is performed according to the cell nucleus classification results, and candidate image blocks corresponding to tumor cell nucleus count rankings greater than a preset threshold are selected as key image blocks.

4. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 3, characterized in that: The step of performing cell channel decoupling on the key image blocks and extracting cell-level pathological omics manual features of different channels for aggregation processing to obtain a patient-level feature vector specifically includes: According to the results of cell nucleus segmentation and classification, the key image blocks are decoupled into cell channels, and 5 cell channel masks are obtained from each key image block; Extract the pathological omics manual features of the corresponding types of cell nuclei of all key image blocks under the masks of the five cell channels to form the cell-level feature matrix of the five separate cell channels; The mean eigenvector and variance eigenvector corresponding to the cell-level feature matrix of different cell channels are obtained respectively, and vector concatenation processing is performed to realize feature aggregation, so as to obtain a patient-level feature vector with several cell channels.

5. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 4, characterized in that: The cell-level pathological omics manual features include cell nucleus shape features, cell nucleus texture features, cell co-occurrence features and cell density features, wherein: The cell nucleus shape feature is obtained by calculating the basic morphology of the cell nucleus, and the cell nucleus shape feature is used to quantify the complexity of the cell nucleus contour and regional attributes; The cell nucleus texture feature is obtained by calculating the first-order statistics of pixel intensity in the cell nucleus region and the second-order statistics based on the gray-level co-occurrence matrix, and the cell nucleus texture feature is used to quantify the complexity of the cell nucleus texture; The cell co-occurrence feature is obtained by calculating the probability that the cell nucleus is predicted to be 5 different cell nucleus types, and the cell co-occurrence feature is used to quantify the degree of cell nucleus heteromorphism or the transition state between different cells; The cell density feature is obtained by calculating the number of cells in all key image blocks in the full image and dividing it by the global tissue region area. The cell density feature is used to quantify the cellular components of the tumor microenvironment.

6. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 5, characterized in that: The step of dividing the patient-level feature vector into a training set and a test set, performing feature selection and splicing and fusion processing based on the training set, performing model training, selecting important features of each channel in the training set, pruning and splicing and fusion processing on the feature matrix of the test set, constructing a single-channel optimal model and a multi-channel optimal fusion model, and determining a single-channel important feature set and a fusion feature set specifically includes: Based on the training phase, the patient-level feature vectors having several cell channels are respectively formed into a patient-level feature matrix for each cell channel; The Lasso regression feature selection algorithm is used to select features from the patient-level feature matrices of different cell channels to obtain several important discriminative features in each channel. The corresponding patient-level important feature submatrices of each channel are input into the corresponding single-channel model for training; The important features selected from all channels are concatenated and fused, and after feature selection, several more discriminative fusion features are obtained. The corresponding fusion feature matrix is ​​input into the multi-channel fusion model for training; Based on the test phase, the patient-level feature vectors of several cell channels are respectively composed into the patient-level feature matrix of each cell channel, and pruned according to the important features of each channel selected in the training phase, and the corresponding important feature sub-matrices are input into the corresponding single-channel model for performance evaluation to build the single-channel optimal model; The patient-level feature matrices of all channels are pruned and concatenated according to the fusion features selected in the training phase. The obtained fusion feature matrix is ​​input into the multi-channel fusion model for performance evaluation to build the multi-channel optimal fusion model. The hyperparameters of each single-channel model and fusion model are adjusted through ten-fold cross validation, and the important features of different channels and fusion features are scored. The optimal model of each single channel and the optimal fusion model are saved, and the important feature sets and fusion feature sets of each single channel with a score ranking higher than the preset ranking are determined.

7. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 6, characterized in that: The fusion feature set includes texture features, shape features and cell co-occurrence features, wherein: The texture features include the range of variance-aggregated tumor cell nucleus correlation, the kurtosis of variance-aggregated tumor cell nucleus pixel intensity, the skewness of variance-aggregated tumor cell nucleus pixel intensity, and the range of variance-aggregated average contrast of other types of cell nuclei; The shape characteristics include the standard deviation of the variance-aggregated area of ​​other types of cell nuclei and the average aggregated center-to-center distance of the adherent normal cell nuclei; The cell co-occurrence features include the average probability that aggregated cohesive normal cell nuclei are predicted to be other types of cell nuclei.

8. A method for predicting BRAF mutation in papillary thyroid carcinoma according to claim 7, characterized in that: The step of inputting each single-channel important feature set into the corresponding single-channel optimal model for reasoning and inputting the fusion feature set into the multi-channel optimal fusion model for reasoning, and voting all model results according to preset weights to obtain the BRAF mutation prediction result specifically includes: Based on the deployment phase, the patient-level feature vectors of each channel of the new patient are obtained, and pruned according to the important feature set of the single channel to obtain the corresponding important feature sub-vectors and input them into the single channel optimal model for reasoning to obtain the first BRAF mutation prediction probability; The patient-level feature vectors of all channels are pruned and concatenated according to the fusion feature set determined in the modeling stage, and the fusion feature vector is input into the multi-channel optimal fusion model for reasoning to obtain the second BRAF mutation prediction probability; According to the first BRAF mutation prediction probability and the second BRAF mutation prediction probability, weights are assigned and voted to obtain the BRAF mutation prediction result.

9. A system for predicting BRAF mutation in papillary thyroid carcinoma, characterized in that: Includes the following modules: The first module is used to obtain the full-frame fine needle aspiration cytology image of papillary thyroid carcinoma and perform quality control, segmentation, initial screening and staining standardization to obtain standardized candidate image blocks; The second module is used to perform cell nucleus segmentation, classification and secondary screening on the standardized candidate image blocks to obtain key image blocks; The third module is used to decouple the key image blocks into cell channels and extract the cell-level pathological omics manual features of different channels for aggregation processing to obtain the patient-level feature vector; The fourth module is used to divide the patient-level feature vector into a training set and a test set, perform feature selection and splicing and fusion processing based on the training set, perform model training, select important features of each channel in the training set, perform pruning and splicing and fusion processing on the feature matrix of the test set, build a single-channel optimal model and a multi-channel optimal fusion model, and determine the important feature set of a single channel and the fusion feature set; The fifth module is used to input the important feature sets of each single channel into the corresponding single-channel optimal model for reasoning and input the fusion feature set into the multi-channel optimal fusion model for reasoning, and vote all model results according to the preset weights to obtain the BRAF mutation prediction results.

Images