A prediction system for the spectrum and subtypes of pituitary neuroendocrine tumors

Abstract

A prediction system for the spectrum and subtype of pituitary neuroendocrine tumors includes an image acquisition module, a spectrum diagnosis module, a biomarker diagnosis module and an expression export module. The image acquisition module is used to generate target data; the spectrum diagnosis module is used to input the target data into the encoder module, generate a feature embedding with high representation ability, and obtain the spectrum category of the target data; the biomarker diagnosis module includes seven biomarker diagnosis subunits, and the seven biomarker diagnosis subunits are used to predict the expression levels of seven biomarkers related to the subtypes of pituitary neuroendocrine tumors; the biomarker diagnosis subunits include a feature classification module, a feature screening module, a feature aggregation module and a biomarker prediction module; the expression export module further analyzes the expression levels of the seven biomarkers on the basis of the spectrum category to obtain the spectrum and subtype results of the pituitary neuroendocrine tumors of the target data.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and more specifically to a prediction system for pituitary neuroendocrine tumor lineages and subtypes. Background Art

[0002] Pituitary neuroendocrine tumors (PitNETs), also known as pituitary adenomas, originate from the anterior pituitary cells and account for about 10% to 20% of primary intracranial tumors, with an incidence rate second only to gliomas and meningiomas. In 2017, the World Health Organization (WHO) promulgated the 4th edition of the pathological classification of endocrine organ-related tumors, revised the pathological classification and nomenclature of pituitary and sellar region tumors, and not only emphasized the renaming of pituitary adenomas, but also proposed to further improve transcription factors as important markers of cell differentiation sources based on immunohistochemistry (Immunohistochemistry, IHC) of hormone secretion by the pituitary gland. In the 5th edition of WHO in 2022, PitNETs / pituitary adenomas are divided into four major lineages and 16 subtypes based on the differentiation lineage of tumor cells: pituitary-specific transcription factor-1 (PIT-1) cell lineage, T-box pituitary transcription factor 19 (TPIT) cell lineage, steroidogenic factor-1 (SF-1) cell lineage, and no definite cell lineage.

[0003] A large number of IHC tests are the basis for PitNETs / pituitary adenoma classification, but they are expensive for patients, and some grassroots units are not yet able to carry out this testing technology. In addition, the use of IHC methods is relatively time-consuming and tissue-consuming, and is easily affected by pre-test conditions. At the same time, there are subjective differences in interpretation between different units and observers or within observers, resulting in high inconsistency in results. Therefore, a new technology is needed to replace the interpretation of pathologists, improve the accuracy of diagnosis, reduce patients' medical costs, and provide more accurate assistance for the individualized and multimodal treatment of PitNETs / pituitary adenoma patients after surgery.

[0004] Artificial intelligence deep learning algorithms can directly extract deep feature information from images and learn automatically. However, the current application of deep learning algorithm models is mostly concentrated in the fields of breast cancer, prostate cancer, colorectal cancer, and head tumors. There is currently no deep learning algorithm for pituitary neuroendocrine tumors. Therefore, it is of great significance to develop a highly accurate artificial intelligence-assisted diagnosis model based on the prediction spectrum and subtypes of pituitary neuroendocrine tumors. Summary of the invention

[0005] The present application is proposed to solve the above problems. According to one aspect of the present application, a prediction system for the lineage and subtypes of pituitary neuroendocrine tumors is provided, including an image acquisition module, a lineage diagnosis module, a biomarker diagnosis module and an expression export module:

[0006] The image acquisition module is used to generate target data;

[0007] The pedigree diagnosis module is used to input the target data into the encoder module, generate feature embeddings with high representation ability, and obtain the pedigree category of the target data;

[0008] The biomarker diagnosis module includes seven biomarker diagnosis subunits, which respectively predict the expression levels of seven biomarkers related to pituitary neuroendocrine tumor subtypes;

[0009] Among them, the biomarker diagnosis subunit includes a feature classification module, a feature screening module, a feature aggregation module and a biomarker prediction module;

[0010] The feature classification module is used to input the target data and obtain the primary classification feature set;

[0011] The feature screening module is used to screen out a feature set for a certain biomarker from the primary classification feature set;

[0012] The feature aggregation module is used to perform weighted fusion on the features in the feature set to obtain a fusion feature for a certain biomarker;

[0013] The biomarker prediction module is used to input the fusion features into the pre-built biomarker prediction network to obtain N classification prediction results for a certain biomarker, where N corresponds to the number of types of seven biomarker expression levels, and then obtain the expression level of a certain biomarker; wherein the seven biomarker diagnostic subunits are independent of each other and have no correlation;

[0014] The expression export module further analyzes the expression levels of seven biomarkers based on the lineage categories to obtain the pituitary neuroendocrine tumor lineage and subtype results of the target data.

[0015] In one embodiment of the present application, each biomarker diagnosis subunit is composed of a feature classification module, a feature screening module, a feature aggregation module and a biomarker prediction module:

[0016] The feature classification module uses the K-means clustering algorithm on the image blocks of the target data to cluster the image blocks with the same features into a class of image block feature sets, and selects the number of classifications by the silhouette coefficient. , is a positive integer, The image patch set is input into the pre-built image patch primary feature extraction network to obtain The primary classification feature set of image patches of categories :

[0017] ;

[0018] is the feature set of each type of image block; is a positive integer;

[0019] The feature screening module compares each type of image block feature set The prediction accuracy of biomarker prediction network input alone is comparable to that of the The primary classification feature set of image patches of categories The average prediction accuracy of all input biomarker prediction networks is used to screen out feature sets with prediction accuracy higher than the average prediction accuracy, and the screened feature set for a certain biomarker is obtained. , used to match a biomarker with the screened image patch feature set:

[0020] ;

[0021] in, is a Boolean operation; It is a primary feature processing network, which is used to extract and compress the input primary feature embedding to obtain the final feature embedding for classification; Used to process the final feature embedding to obtain the final prediction score for each classification category;

[0022] The feature aggregation module aggregates the feature set obtained by the feature screening module The features are normalized, the features are re-weighted, and the obtained weights are combined with the input feature set of each image block Perform residual calculation to obtain weighted features, and concatenate the weighted features to obtain attention-based feature weighted fusion features. :

[0023] ;

[0024] in, for Operation, used to normalize the input features to ensure that each feature is re-weighted in the range of [0, 1]; Used to combine the obtained weights with the input image block feature set of each type Perform residual calculation to obtain weighted features; The number of image block categories filtered by the feature filtering module (n <m), is the feature set of each type of image block, is the filtered feature set, and are learnable parameters;

[0025] The biomarker prediction module is used to convert the fusion feature By inputting into the pre-built biomarker prediction network, N classification prediction results for a certain biomarker can be obtained, where N corresponds to the number of types of expression levels of the seven biomarkers.

[0026] In one embodiment of the present application, the pedigree diagnosis module is used to input the target data into the encoder module, generate a feature embedding with high representation ability, and obtain the pedigree category of the target data, including:

[0027] The target data is input into the pre-built primary neural network model to obtain the primary features of the target data;

[0028] Inputting the primary features of the target data into four consecutive encoder blocks to obtain feature embeddings with high representation capabilities;

[0029] After embedding the features with high representation ability for dimensionality reduction compression, The function calculates the prediction score of each lineage, and the one with the highest score is the lineage category corresponding to the target data.

[0030] In one embodiment of the present application, the operation of the encoder block is:

[0031] ;

[0032] ;

[0033] in, for The output feature embedding of the encoder block, is the primary image feature of the target data, For the Feature attention maps of encoder blocks, is a feed-forward neural network, is the layer normalization process, is a multi-head self-attention model used to compute the feature attention map.

[0034] In one embodiment of the present application, an auxiliary enhancement module is also included:

[0035] The auxiliary enhancement module is used to balance the imbalance of target data ratio by using reference data during the training process, and to assist the pedigree diagnosis module in parameter optimization by sharing model parameters;

[0036] The reference data has similar numbers of four pedigree label types, which are used to improve the pedigree diagnosis module's ability to identify pedigrees with fewer numbers.

[0037] In one embodiment of the present application, the auxiliary enhancement module has the same structure as the pedigree diagnosis module, and the reference data is input into the auxiliary enhancement module for training. The parameter optimization in the pedigree diagnosis module is achieved by calculating the mean square error loss of the feature attention maps in the first two encoder blocks in the two models of the auxiliary enhancement module and the pedigree diagnosis module.

[0038] In one embodiment of the present application, the mean square error loss is calculated as follows:

[0039] ;

[0040] in, Represents the pedigree diagnosis model, stands for auxiliary enhancement model, The target data is in the auxiliary enhancement model The feature attention map generated in the encoder block, The reference data is used in the auxiliary enhancement model. The feature attention map generated in the encoder block, To calculate the mean squared error between feature attention maps in the same encoder block.

[0041] In one embodiment of the present application, the target data is a <pituitary neuroendocrine tumor category label, data> pair consisting of the spectrum and subtype classification results of pituitary neuroendocrine tumors as category information and corresponding target data.

[0042] In one embodiment of the present application, the target data is obtained by segmenting and screening the pathological section image through a preset grayscale threshold.

[0043] In one embodiment of the present application, a pre-built neural network model uses mean square error loss and cross entropy loss functions for training and learning, wherein the pre-built neural network model includes a pre-trained primary neural network model, an encoder module, an image block primary feature extraction network, a primary feature processing network, and a biomarker prediction network.

[0044] The prediction system of the pituitary neuroendocrine tumor lineage and subtype of the present invention includes: an image acquisition module, a lineage diagnosis module, a biomarker diagnosis module and an expression export module. The image acquisition module generates target data containing pituitary neuroendocrine tumors, and the lineage diagnosis module generates feature embedding with high representation ability to obtain the lineage category of the target data. The biomarker diagnosis module predicts the expression levels of seven biomarkers related to pituitary neuroendocrine tumor subtypes. The expression export module further analyzes the expression levels of the seven biomarkers based on the lineage category to obtain the pituitary neuroendocrine tumor lineage and subtype results corresponding to the pathological section image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0046] Figure 1 A system block diagram of a system for predicting pituitary neuroendocrine tumor lineages and subtypes according to an embodiment of the present application is shown;

[0047] Figure 2 A system block diagram of a biomarker diagnosis subunit in a prediction system for pituitary neuroendocrine tumor lineages and subtypes according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application more obvious, the example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present application.

[0049] First, refer to Figure 1 To describe a prediction system for pituitary neuroendocrine tumor lineages and subtypes for implementing an embodiment of the present invention. Figure 1 The system block diagram of the prediction system of pituitary neuroendocrine tumor lineages and subtypes according to the embodiment of the present application is shown. The prediction system of pituitary neuroendocrine tumor lineages and subtypes includes an image acquisition module, a lineage diagnosis module, a biomarker diagnosis module and an expression export module.

[0050] The image acquisition module is used to scan the patient's pathological slice images to generate target data containing pituitary neuroendocrine tumors. The pedigree diagnosis module is used to process the target data, input the primary features of the target data into the encoder module, generate feature embeddings with high representation capabilities, and then obtain the pedigree category of the target data. The biomarker diagnosis module includes seven biomarker diagnosis subunits, which are used to predict the expression levels of seven biomarkers related to pituitary neuroendocrine tumor subtypes through the seven biomarker diagnosis subunits.

[0051] Specifically, Figure 2 As shown, Figure 2 A system block diagram of a biomarker diagnosis subunit in a prediction system for the lineage and subtype of pituitary neuroendocrine tumors provided in an embodiment of the present application. Among them, the biomarker diagnosis subunit includes a feature classification module, a feature screening module and a biomarker prediction module. The feature classification module is used to input the target data to obtain a primary classification feature set. The feature screening module is used to filter out a feature set for a certain biomarker from the primary classification feature set. The feature aggregation module is used to perform weighted fusion of the features in the feature set to obtain a fusion feature for a certain biomarker. The biomarker prediction module is used to input the fusion feature into a pre-constructed biomarker prediction network to obtain N classification prediction results for a certain biomarker, where N corresponds to the number of types of seven biomarker expression levels, and then the expression level for a certain biomarker can be obtained; wherein, the seven biomarker diagnosis subunits are independent of each other and have no correlation.

[0052] Based on the lineage categories, the expression export module further analyzes the expression levels of seven biomarkers to obtain the pituitary neuroendocrine tumor lineage and subtype results corresponding to the pathological section images.

[0053] The prediction system of the pituitary neuroendocrine tumor lineage and subtype of the present invention includes: an image acquisition module, a lineage diagnosis module, a biomarker diagnosis module and an expression export module. The image acquisition module generates target data containing pituitary neuroendocrine tumors, and the lineage diagnosis module generates feature embedding with high representation ability to obtain the lineage category of the target data. The biomarker diagnosis module predicts the expression levels of seven biomarkers related to pituitary neuroendocrine tumor subtypes. The expression export module further analyzes the expression levels of the seven biomarkers on the basis of the lineage category to obtain the pituitary neuroendocrine tumor lineage and subtype results corresponding to the pathological section image.

[0054] Because the system contains many neural network models, the neural network models need to go through a training process before actual operation. The training process is described in detail below.

[0055] In one embodiment, the target data is a <pituitary neuroendocrine tumor category label, data> pair consisting of the category information and the corresponding target data, and the result of the classification of the lineage and subtype of the pituitary neuroendocrine tumor.

[0056] The patient's pathological section images are scanned through the image acquisition module to obtain the <pituitary neuroendocrine tumor category label, data> pair, thereby generating the training set and test set. The data is the digital pathological image (whole slide image, WSI) obtained after scanning each pathological section image, and the pituitary neuroendocrine tumor category label is the lineage and subtype category corresponding to the WSI. The above WSI and the lineage and subtype categories corresponding to the WSI are data provided by the hospital. The target data in the training process is the <pituitary neuroendocrine tumor category label, data> pair obtained by the above processing, and the purpose is to train the system through the target data.

[0057] In one embodiment, the target data is obtained by segmenting and screening the pathological slice image through a preset grayscale threshold.

[0058] After obtaining the target data, the WSI in the target data needs to be preprocessed. The WSI preprocessing operation is to divide the complete WSI into several image blocks (patches). The specific operation is to segment and filter the WSI by a preset grayscale threshold to obtain patches.

[0059] In one embodiment, the pedigree diagnosis module is used to process the target data, input the primary features of the target data into the encoder module, generate feature embeddings with high representation capabilities, and then obtain the pedigree category of the target data, including: the target data is input into a pre-built primary neural network model to obtain the primary features of the target data.

[0060] The primary features of the target data are input into four consecutive encoder blocks to obtain feature embeddings with high representation capabilities. All four encoder blocks are Transformer encoders, a new type of encoder that has been widely used in recent years. It has the ability to extract useful information from primary features and can obtain feature embeddings with high representation capabilities. The series combination of multiple Transformer encoders can further improve the ability to extract features and have the same organizational structure.

[0061] After the feature embedding with high representation ability is compressed by dimensionality reduction, The function calculates the prediction score of each lineage, and the one with the highest score is the lineage category corresponding to the target data. The four lineages are: PIT-1, TPIT, SF-1 and no lineage. The dimensionality reduction compression process is the pooling process in the neural network. Function is the function in the classification task.

[0062] In an embodiment of the present application, the operation of the encoder block is:

[0063] ;

[0064] ;

[0065] in, for The output feature embedding of the encoder block, is the primary image feature of the target data, For the Feature attention maps of encoder blocks, is a feed-forward neural network, is the layer normalization process, is a multi-head self-attention model used to compute the feature attention map.

[0066] In one embodiment, an auxiliary enhancement module is also included: the auxiliary enhancement module is used to balance the imbalance of the target data ratio by using reference data during the training process, and to assist the pedigree diagnosis module in parameter optimization by sharing model parameters, thereby further improving the prediction ability of the target data. The number of the four pedigree label types in the reference data is similar, which is used to improve the recognition ability of the pedigree diagnosis module for pedigrees with a smaller number.

[0067] In one embodiment, the auxiliary enhancement module has the same structure as the pedigree diagnosis module, and reference data is input into the auxiliary enhancement module for training. By calculating the mean square error loss of the feature attention maps in the first two encoder blocks in the two models of the auxiliary enhancement module and the pedigree diagnosis module, parameter optimization in the pedigree diagnosis module is achieved, further enhancing the accuracy of the pedigree diagnosis module in extracting target data feature embedding and improving the ability to predict the pedigree category of the target data.

[0068] The ratio of the four lineages in the target data is 4:2:1:1, and there is an imbalance in the number of lineage label types. If only the target data is used to train the neural network model in the system, the model's ability to recognize lineages with a smaller number will be reduced, thereby weakening the system's overall classification ability for pituitary neuroendocrine tumor tissue pathology images.

[0069] Therefore, an auxiliary enhancement module is introduced in the training process to solve the above problems. The reference data is the <pituitary neuroendocrine tumor category label, data> pair other than the target data. It can be a publicly available online data set such as TCIA. The characteristics of the reference data are that it has a huge number base in various pedigrees. Therefore, four pedigrees with similar numbers of reference data are obtained by selection. The auxiliary enhancement module is an independent system. Its system structure is exactly the same as that of the pedigree diagnosis module. The only difference is that the data used for model training is different. The input of the former is reference data, while the latter is target data.

[0070] In one embodiment, the mean square error loss is calculated as:

[0071] ;

[0072] in, Represents the pedigree diagnosis model, stands for auxiliary enhancement model, The target data is in the auxiliary enhancement model The feature attention map generated in the encoder block, The reference data is used in the auxiliary enhancement model. The feature attention map generated in the encoder block, To calculate the mean squared error between feature attention maps in the same encoder block.

[0073] The features extracted by the shallow Transformer encoder tend to be specific features between different lineages, while the features extracted by the deep Transformer encoder tend to be refined features under a specific lineage. The original intention of introducing auxiliary enhancement is to better distinguish the differences between lineages. Therefore, only the first two shallow Transformer encoders are used to calculate the mean square error loss and optimize the corresponding parameters.

[0074] At present, the classification of pituitary adenomas mainly relies on immunohistochemistry (IHC). In addition to the main transcription factors PIT1, TPIT, SF1 and the secondary transcription factors GATA3 and ER, it also includes six hormones ACTH, GH, PRL, TSH, FSH, LH and cytokeratin CK8 / 18 and cell proliferation index Ki-67. The expression levels of the seven biomarkers are as follows: GH, TSH, FSH, LH: negative / positive, PRL: negative / positive / paranuclear dot positive, ACTH: diffuse positive / scattered positive / weakly positive, CK8 / 18: scattered paranuclear dot positive / paranuclear dot positive / circle positive / diffuse positive / perinuclear positive / nuclear positive / weakly positive / positive / focal + / negative.

[0075] The subtype diagnosis of pituitary neuroendocrine tumors requires further analysis of the expression levels of the above seven biomarkers based on the known pituitary tumor lineage of the patient. Therefore, the function of the biomarker diagnosis module is to determine the expression levels of the seven biomarkers.

[0076] The diagnosis of pituitary tumor subtypes is related to the expression levels of seven biomarkers. Since there are many types of biomarkers and their expressions are different, it is difficult to detect the expression levels of seven biomarkers at the same time. Seven independent biomarker detection subunits are used to independently detect the expression level of each biomarker. Each biomarker detection subunit is basically the same in structure and is divided into a feature classification module, a feature screening module and a biomarker prediction module.

[0077] In one embodiment, each biomarker diagnosis subunit is composed of a feature classification module, a feature screening module, a feature aggregation module and a biomarker prediction module:

[0078] The feature classification module uses the K-means clustering algorithm on the image blocks of the target data to cluster the image blocks with the same features into a class of image block feature sets, and selects the number of classifications by the silhouette coefficient. , is a positive integer, The image patch set is input into the pre-built image patch primary feature extraction network to obtain The primary classification feature set of image patches of categories :

[0079] ;

[0080] is the feature set of each type of image block; is a positive integer;

[0081] The feature filtering module compares the feature set of each type of image block The prediction accuracy of biomarker prediction network input alone is comparable to that of the The primary classification feature set of image patches of categories The average prediction accuracy of all input biomarker prediction networks is used to screen out feature sets with prediction accuracy higher than the average prediction accuracy, and the screened feature set for a certain biomarker is obtained. , used to match a biomarker with the screened image patch feature set:

[0082] ;

[0083] in, is a Boolean operation; It is a primary feature processing network, which is used to extract and compress the input primary feature embedding to obtain the final feature embedding for classification; Used to process the final feature embedding to obtain the final prediction score for each classification category;

[0084] The feature aggregation module aggregates the features obtained by the feature screening module The features are normalized, the features are re-weighted, and the obtained weights are combined with the input feature set of each image block Perform residual calculation to obtain weighted features, and concatenate the weighted features to obtain attention-based feature weighted fusion features. :

[0085] ;

[0086] in, for Operation, used to normalize the input features to ensure that each feature is re-weighted in the range of [0, 1]; Used to combine the obtained weights with the input image block feature set of each type Perform residual calculation to obtain weighted features; The number of image block categories filtered by the feature filtering module (n <m), is the feature set of each type of image block, is the filtered feature set, and are learnable parameters;

[0087] The biomarker prediction module is used to integrate the fusion features By inputting into the pre-built biomarker prediction network, N classification prediction results for a certain biomarker can be obtained, where N corresponds to the number of types of expression levels of the seven biomarkers.

[0088] In one embodiment, the pre-built neural network model uses mean square error loss and cross entropy loss functions for training and learning, wherein the pre-built neural network model includes a pre-trained primary neural network model, an encoder module, an image block primary feature extraction network, a primary feature processing network, and a biomarker prediction network.

[0089] The difference between the system operation process and the training process is that the target data of the operation process is a digital pathology image of a pituitary tumor of unknown lineage and subtype category, and the output of the system is the predicted lineage and predicted subtype category of the digital pathology image.

[0090] Compared with the training process, the running process lacks an auxiliary enhancement module. The reason is that the role of the auxiliary enhancement module is to improve the performance of the system during the training process. The running process is a trained system, so this module is no longer needed. The remaining modules are retained. The specific functions of each module are consistent with the functions in the training process, and no redundant elaboration is given here.

[0091] On the other hand, the present application also provides a method for predicting the lineage and subtype of pathological slice images of pituitary neuroendocrine tumors, which specifically includes the following steps:

[0092] S1: Scanning the patient's pathological slice image through the image acquisition module to generate target data containing pituitary neuroendocrine tumors;

[0093] S2: The target data is processed through the pedigree diagnosis module, the primary features of the target data are input into the encoder module, and feature embedding with high representation ability is generated, thereby obtaining the pedigree category of the target data;

[0094] S3: The expression levels of seven biomarkers associated with pituitary neuroendocrine tumor subtypes were predicted by the seven biomarker diagnostic subunits of the biomarker diagnostic module.

[0095] The step S3 also includes:

[0096] S31: clustering the image block set after the target data is evenly divided through the feature classification module, and inputting it into the pre-designed image block primary feature extraction network to obtain the feature set corresponding to each type of image block;

[0097] S32: By comparing the prediction accuracy of each type of feature with that of all features through the feature screening module, a feature set for a certain biomarker can be screened out;

[0098] S33: performing weighted fusion on the features in the feature set through a feature aggregation module to obtain a fusion feature for a certain biomarker;

[0099] S34: The fused features are input into a pre-built biomarker prediction network to obtain a binary classification prediction result for a certain biomarker.

[0100] S4: Based on the spectrum categories, the expression levels of seven biomarkers were further analyzed to obtain the spectrum and subtype results of pituitary neuroendocrine tumors corresponding to the pathological section images.

[0101] The prediction system of the pituitary neuroendocrine tumor lineage and subtype of the present invention includes: an image acquisition module, a lineage diagnosis module, a biomarker diagnosis module and an expression export module. The image acquisition module generates target data containing pituitary neuroendocrine tumors, and the lineage diagnosis module generates feature embedding with high representation ability to obtain the lineage category of the target data. The biomarker diagnosis module predicts the expression levels of seven biomarkers related to pituitary neuroendocrine tumor subtypes. The expression export module further analyzes the expression levels of the seven biomarkers on the basis of the lineage category to obtain the pituitary neuroendocrine tumor lineage and subtype results corresponding to the pathological section image.

[0102] In addition, the present application also provides a storage medium having a computer program stored thereon, and when the computer program is run by a processor, the processor executes the prediction system for the lineage and subtype of pituitary neuroendocrine tumors according to the embodiment of the present application as described above. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0103] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0104] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0106] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0107] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present application should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0108] Those skilled in the art will understand that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstract and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0109] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0110] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The present application can also be implemented as a program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0111] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" before an element does not exclude the presence of multiple such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

[0112] The above is only a specific implementation or description of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A prediction system for the lineage and subtypes of pituitary neuroendocrine tumors, characterized in that: Including image acquisition module, pedigree diagnosis module, biomarker diagnosis module and expression export module: The image acquisition module is used to generate target data; The pedigree diagnosis module is used to input the target data into the encoder module, generate feature embeddings with high representation ability, and obtain the pedigree category of the target data; The biomarker diagnosis module includes seven biomarker diagnosis subunits, which respectively predict the expression levels of seven biomarkers related to pituitary neuroendocrine tumor subtypes; Among them, the biomarker diagnosis subunit includes a feature classification module, a feature screening module, a feature aggregation module and a biomarker prediction module; The feature classification module is used to input the target data and obtain the primary classification feature set; The feature screening module is used to screen out a feature set for a certain biomarker from the primary classification feature set; The feature aggregation module is used to perform weighted fusion on the features in the feature set to obtain a fusion feature for a certain biomarker; The biomarker prediction module is used to input the fusion features into the pre-built biomarker prediction network to obtain N classification prediction results for a certain biomarker, where N corresponds to the number of types of seven biomarker expression levels, and then obtain the expression level of a certain biomarker; wherein the seven biomarker diagnostic subunits are independent of each other and have no correlation; The expression export module further analyzes the expression levels of seven biomarkers based on the lineage categories to obtain the pituitary neuroendocrine tumor lineage and subtype results of the target data.

2. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 1, characterized in that: Each biomarker diagnosis subunit consists of a feature classification module, a feature screening module, a feature aggregation module, and a biomarker prediction module. composition: The feature classification module uses the K-means clustering algorithm on the image blocks of the target data to cluster the image blocks with the same features into a class of image block feature sets, and selects the number of classifications by the silhouette coefficient. , is a positive integer, The image patch set is input into the pre-built image patch primary feature extraction network to obtain The primary classification feature set of image patches of categories : ； is the feature set of each type of image block; is a positive integer; The feature screening module compares each type of image block feature set The prediction accuracy of biomarker prediction network input alone is comparable to that of the The primary classification feature set of image patches of categories The average prediction accuracy of all input biomarker prediction networks is used to screen out feature sets with prediction accuracy higher than the average prediction accuracy, and the screened feature set for a certain biomarker is obtained. , used to match a biomarker with the screened image patch feature set: ； in, is a Boolean operation; It is a primary feature processing network, which is used to extract and compress the input primary feature embedding to obtain the final feature embedding for classification; Used to process the final feature embedding to obtain the final prediction score for each classification category; The feature aggregation module aggregates the feature set obtained by the feature screening module The features are normalized, the features are re-weighted, and the obtained weights are combined with the input feature set of each image block Perform residual calculation to obtain weighted features, and concatenate the weighted features to obtain attention-based feature weighted fusion features. : ； in, for Operation, used to normalize the input features to ensure that each feature is re-weighted in the range of [0, 1]; Used to combine the obtained weights with the input image block feature set of each type Perform residual calculation to obtain weighted features; The number of image block categories filtered by the feature filtering module (n <m), is the feature set of each type of image block, is the filtered feature set, and are learnable parameters; The biomarker prediction module is used to convert the fusion feature By inputting into the pre-built biomarker prediction network, N classification prediction results for a certain biomarker can be obtained, where N corresponds to the number of types of expression levels of the seven biomarkers.

3. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 1, characterized in that: The pedigree diagnosis module is used to input the target data into the encoder module, generate feature embedding with high representation ability, and obtain the pedigree category of the target data, including: The target data is input into the pre-built primary neural network model to obtain the primary features of the target data; Inputting the primary features of the target data into four consecutive encoder blocks to obtain feature embeddings with high representation capabilities; After embedding the features with high representation ability for dimensionality reduction compression, The function calculates the prediction score for each lineage, where the one with the highest score is the lineage category corresponding to the target data.

4. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 3, characterized in that: The operations of the encoder block are: ； ； in, for The output feature embedding of the encoder block, is the primary image feature of the target data, For the The feature attention map of the encoder block, is a feed-forward neural network, is the layer normalization process, is a multi-head self-attention model used to compute the feature attention map.

5. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 4, characterized in that: Also includes auxiliary enhancement modules: The auxiliary enhancement module is used to balance the imbalance of target data ratio by using reference data during the training process, and to assist the pedigree diagnosis module in parameter optimization by sharing model parameters; The reference data has similar numbers of four pedigree label types, which are used to improve the pedigree diagnosis module's ability to identify pedigrees with fewer numbers.

6. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 5, characterized in that: The auxiliary enhancement module has the same structure as the pedigree diagnosis module. The reference data is input into the auxiliary enhancement module for training. The parameter optimization in the pedigree diagnosis module is achieved by calculating the mean square error loss of the feature attention maps in the first two encoder blocks in the two models of the auxiliary enhancement module and the pedigree diagnosis module.

7. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 6, characterized in that: The calculation method of the mean square error loss is: ； in, Represents the pedigree diagnosis model, stands for auxiliary enhancement model, The target data is in the auxiliary enhancement model The feature attention map generated in the encoder block, The reference data is used in the auxiliary enhancement model. The feature attention map generated in the encoder block, To calculate the mean squared error between feature attention maps in the same encoder block.

8. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 1, characterized in that: The target data is a pair of <pituitary neuroendocrine tumor category label, data> consisting of the spectrum and subtype classification results of pituitary neuroendocrine tumors as category information and the corresponding target data.

9. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 1, characterized in that: The target data is obtained by segmenting and screening the pathological slice image through a preset grayscale threshold.

10. The prediction system for pituitary neuroendocrine tumor lineages and subtypes according to claim 1, characterized in that: The pre-built neural network model uses mean square error loss and cross entropy loss functions for training and learning, wherein the pre-built neural network model includes a pre-trained primary neural network model, an encoder module, an image block primary feature extraction network, a primary feature processing network, and a biomarker prediction network.

Images