Bladder cancer gene mutation prediction system based on deep learning

Abstract

The invention provides a bladder cancer gene mutation prediction system based on deep learning, and the system comprises a histopathologic image collection module which is used for collecting an original to-be-detected histopathologic image; the preprocessing module is used for preprocessing the original to-be-detected histopathologic image by adopting an HE staining method and a full-slide imaging scanner to obtain WSI with HE stained bladder tissue; the cloud server module is used for calling a bladder cancer deep learning prediction model to perform bladder cancer gene expression state prediction, bladder cancer gene methylation state prediction and bladder cancer gene mutation state prediction on the WSI to generate corresponding prediction results; and the report generation module is used for automatically generating a bladder cancer gene mutation prediction auxiliary diagnosis report according to the corresponding prediction result and feeding back the report to the specified terminal.

Description

Technical Field

[0001] The present invention relates to the technical field of auxiliary diagnosis of bladder cancer, and in particular to a bladder cancer gene mutation prediction system based on deep learning. Background Art

[0002] Bladder cancer is the most common malignant tumor of the urinary system and ranks ninth among the most common tumors in the world [PMID] 36633525 [PMID] 33433946. It ranks first in the incidence of urogenital tumors in my country, and second in the West after prostate cancer. In 2020, there are an estimated 573,275 new cases and 212,536 deaths worldwide [PMID] 33538338.

[0003] The pathological types of bladder cancer include urothelial carcinoma, squamous cell carcinoma, adenocarcinoma, and other rare types include bladder clear cell carcinoma, bladder small cell carcinoma, and bladder carcinoid. The most common type is bladder urothelial carcinoma, accounting for more than 90% of the total number of bladder cancer patients. Pathological diagnosis is still the gold standard for the diagnosis of bladder cancer, but traditional pathological diagnosis is heavily dependent on the professional knowledge and diagnostic experience of pathologists, and the subjectivity of pathologists leads to inconsistent diagnosis. At the same time, qualitative diagnosis based solely on tissue morphology cannot meet the needs of patients for precise and individualized treatment. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a bladder cancer gene mutation prediction system based on deep learning to solve the above problems.

[0005] A bladder cancer gene mutation prediction system based on deep learning includes: a tissue pathology image acquisition module, used to acquire original tissue pathology images to be tested; a preprocessing module, used to preprocess the original tissue pathology images to be tested using HE staining and a whole slide imaging scanner to obtain WSI with HE-stained bladder tissue; a cloud service module, used to call a bladder cancer deep learning prediction model to predict the bladder cancer gene expression state, bladder cancer gene methylation state and bladder cancer gene mutation state of the WSI, and generate corresponding prediction results; a report generation module, used to automatically generate a bladder cancer gene mutation prediction auxiliary diagnosis report according to the corresponding prediction results, and feed it back to a designated terminal.

[0006] As an embodiment of the present invention, the histopathology image acquisition module is connected to a preset urinary system image acquisition device, and is used to acquire the original histopathology image to be tested acquired by the preset urinary system image acquisition device.

[0007] As an embodiment of the present invention, the preprocessing module performs the following operations: staining the bladder tissue to be tested according to the HE staining method to obtain the HE-stained bladder tissue; and using a whole slide imaging scanner to obtain a WSI of the HE-stained bladder tissue based on the original pathological image of the tissue to be tested after staining by the HE staining method.

[0008] As an embodiment of the present invention, the training process of a deep learning prediction model for bladder cancer includes the following steps: obtaining training samples; constructing a deep learning framework based on a convolutional neural network, and training the deep learning framework according to the training samples until the preset conditions are met to terminate the training, thereby generating a deep learning prediction model for bladder cancer.

[0009] As an embodiment of the present invention, obtaining training samples includes: obtaining training WSIs of several HE-stained bladder tissues from the public TCGA; labeling the training WSIs and filtering the training WSIs to obtain several WSI images that only retain gene expression, gene methylation and gene mutation labels; dividing the several WSI images to obtain a verification WSI set, a training WSI set and a test WSI set; based on a 400-fold magnification, dividing all WSI sets into image blocks of 320×320 pixels and filtering the background to obtain a second verification WSI set, a second training WSI set and a second test WSI set.

[0010] As an embodiment of the present invention, a bladder cancer gene mutation prediction system based on deep learning also includes a two-stage pre-analysis method to accelerate the training process. The first stage acceleration process is: based on a 100-fold magnification, all WSI sets are divided into image blocks of 320×320 pixels, and the background is filtered to obtain a third verification WSI set, a third training WSI set and a third test WSI set for constructing an analysis data set; and a RestNet-50 model pre-trained on ImageNet is used to convert each image block in the analysis data set into a 2048-dimensional vector as a feature extractor.

[0011] As an embodiment of the present invention, the two-stage acceleration process is: using the 3-fold cross validation method in the k-fold cross validation method to evaluate the performance of Xgboost trained on the feature vector to obtain an evaluation result; wherein, in each fold, 200 WSIs in the analysis data set are randomly selected for training, and 100 WSIs are selected for testing; based on the RestNet-50 model, a classification model is rebuilt on the second verification WSI set, the second training WSI set, and the second test WSI set, and detailed classification information is provided for each selected gene for labeling in combination with the evaluation result.

[0012] As an embodiment of the present invention, the report generation module performs the following operations: obtaining the corresponding prediction result, extracting the corresponding original case data from the preset database according to the corresponding prediction result, wherein the original case data is the case data of the patient corresponding to the current prediction result participating in the prediction during this prediction; extracting the valid initial text corpus in the original case data based on the regular expression, wherein each initial text corpus uniquely corresponds to a prediction result; sorting all the initial text corpora according to the text length, counting the text length, and constructing a text length relationship table; performing entity extraction of the medical entity type on each initial text corpus, and determining each initial text corpus. The entity type of the initial text corpus is determined, and according to the preset entity association relationship, the entity types of all the initial text corpora under each prediction result are associated to obtain the entity association relationship of each initial text corpus; the initial report template is determined according to the longest text length in the text length relationship table, and based on the entity association relationship of the initial text corpus under each prediction result, any number of associated initial text corpora are merged and completed to obtain the report text to be generated; based on the initial report text filling logic corresponding to the initial report template, the report text to be generated and the prediction result are matched and filled into the initial report template to generate a bladder cancer gene mutation prediction auxiliary diagnosis report.

[0013] As an embodiment of the present invention, a bladder cancer gene mutation prediction system based on deep learning also includes: obtaining the user diagnostic intention uploaded by the designated terminal, the user diagnostic intention includes various medical subjects, wherein the user diagnostic intention includes multiple; analyzing all the user diagnostic intentions uploaded this time, determining the diagnostic field for which the user needs auxiliary diagnosis this time, removing irrelevant content in the bladder cancer gene mutation prediction auxiliary diagnosis report based on the diagnostic field, and generating a corresponding accurate auxiliary diagnosis report to feed back to the designated terminal.

[0014] As an embodiment of the present invention, all the diagnostic intentions of the users uploaded this time are analyzed to determine the diagnostic field for which the users need auxiliary diagnosis this time, including: establishing a diagnostic intention database, wherein the diagnostic intention database stores a number of diagnostic intentions with marked diagnostic fields, each diagnostic intention is marked with an association value with different diagnostic fields, and each diagnostic intention is related to a number of diagnostic fields at the same time; based on the diagnostic intention database, generating a first association value m between each user's diagnostic intention and different diagnostic fields, wherein each user's diagnostic intention has a first association value m with each diagnostic field, and based on all the first association values ​​m, counting a number of second association values ​​of the user's diagnostic intentions corresponding to any diagnostic field, the second association values ​​including m1, m2...m n, where n is the total number of user diagnostic intentions, all user diagnostic intentions corresponding to the second correlation values ​​exceeding the preset statistical threshold p are screened as target diagnostic intentions, and the number of target diagnostic intentions in each diagnostic field is calculated. i , where i is the number of diagnostic fields; the number of target diagnostic intentions in each diagnostic field is counted k i The proportion of the total number of users' diagnostic intentions i ; Pre-set condition to meet value g, when k i With j i When the product of is not less than g, take k i With j i The diagnostic field corresponding to the maximum value of the product is output as the target diagnostic field to determine the diagnostic field that the user needs auxiliary diagnosis for this time.

[0015] The beneficial effects of the present invention are:

[0016] The present invention provides a bladder cancer gene mutation prediction system based on deep learning. Deep learning (DL) is used to extract pathological morphological features from hematoxylin and eosin (H&E) images that can accurately predict genetic abnormalities in bladder cancer patients, establish the connection between bladder cancer gene changes and pathological morphology, help pathologists to pre- and efficiently determine the gene changes represented by specific pathological morphology, guide the next step of diagnosis and treatment strategy, and solve the problem that the existing qualitative diagnosis based solely on tissue morphology cannot meet the needs of patients for precise and individualized treatment.

[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0018] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0020] Figure 1 Schematic diagram of system modules of a bladder cancer gene mutation prediction system based on deep learning in an embodiment of the present invention;

[0021] Figure 2 A flowchart for generating an accurate auxiliary diagnosis report in a bladder cancer gene mutation prediction system based on deep learning in an embodiment of the present invention;

[0022] Figure 3 A diagnostic field determination flowchart for generating a flowchart in a bladder cancer gene mutation prediction system based on deep learning in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0024] See also Figure 1 A bladder cancer gene mutation prediction system based on deep learning includes: a tissue pathology image acquisition module 1, used to acquire original tissue pathology images to be tested; a preprocessing module 2, used to preprocess the original tissue pathology images to be tested using HE staining and a whole slide imaging scanner to obtain WSI (whole slide images) with HE staining of bladder tissue; a cloud service module 3, used to call a bladder cancer deep learning prediction model to predict the bladder cancer gene expression state, bladder cancer gene methylation state and bladder cancer gene mutation state of the WSI, and generate corresponding prediction results; a report generation module 4, used to automatically generate a bladder cancer gene mutation prediction auxiliary diagnosis report according to the corresponding prediction results, and feed it back to a designated terminal;

[0025] The working principle of the above technical solution is as follows: the present invention provides a bladder cancer gene mutation prediction system based on deep learning, the system includes a histopathology image acquisition module, a preprocessing module, a cloud server module and a report generation module, wherein the histopathology image acquisition module acquires the original histopathology image to be tested; the preprocessing module uses HE staining method and a whole slide imaging scanner to preprocess the original histopathology image to be tested to obtain a WSI with HE-stained bladder tissue; the cloud server module calls the bladder cancer deep learning prediction model to predict the bladder cancer gene expression status, bladder cancer gene methylation status and bladder cancer gene mutation status of the WSI, and generates corresponding prediction results; the report generation module automatically generates a bladder cancer gene mutation prediction auxiliary diagnosis report according to the corresponding prediction results, and feeds it back to the designated terminal;

[0026] The beneficial effects of the above technical solution are: through the above technical solution, deep learning (DL) is used to extract pathological morphological features from hematoxylin and eosin (H&E) images that can accurately predict genetic abnormalities in bladder cancer patients, establish a connection between bladder cancer gene changes and pathological morphology, and help pathologists to pre-emptively and efficiently determine the genetic changes represented by specific pathological morphologies, guide the next step of diagnosis and treatment strategies, and solve the problem that the existing qualitative diagnosis based solely on tissue morphology cannot meet the needs of patients for precise and individualized treatment.

[0027] In one embodiment, the histopathology image acquisition module is connected to a preset urinary system image acquisition device, and is used to acquire the original histopathology image to be tested acquired by the preset urinary system image acquisition device.

[0028] In one embodiment, the preprocessing module performs the following operations: staining the bladder tissue to be tested according to the HE staining method to obtain the HE-stained bladder tissue; and obtaining the WSI of the HE-stained bladder tissue using a whole slide imaging scanner based on the original pathological image of the tissue to be tested after staining by the HE staining method.

[0029] In one embodiment, the training process of the deep learning prediction model for bladder cancer includes the following steps: obtaining training samples; constructing a deep learning framework based on a convolutional neural network, training the deep learning framework according to the training samples, and ending the training when preset conditions are met to generate a deep learning prediction model for bladder cancer;

[0030] It is worth noting that the outputs of the deep learning prediction model for bladder cancer include deep learning prediction of bladder cancer gene expression status, deep learning prediction of bladder cancer gene methylation status, and deep learning prediction of bladder cancer gene mutation status;

[0031] In a specific embodiment, we first used a deep learning framework to predict bladder cancer gene expression status from histopathological images; we collected 14,231 gene expression data matched to each WSI patient; in order to reduce the analysis scope of predictable genes, we performed a two-stage pre-analysis method on all genes; in the cross-validation stage, the performance of the gene was measured by the area under the curve (AUC) of the average sliding level of all three folds; the slope prediction was calculated by averaging the prediction probability of all tiles of a WSI. We trained a model from scratch for each selected gene using ResNet-50 on a 400-fold enlarged dataset to obtain the final prediction results; finally, in the test group, the top ten genes-CALD1, FGF7, ITGA5, PLN, SFRP2, VSIG4, PDLIM3, CYBRD1, SHISAL1 and CPXM2 were predicted from pathological images with AUC ranging from 0.854 to 0.951;

[0032] Then, based on the same method, the methylation status of bladder cancer genes was predicted. In a specific embodiment, 14,231 gene methylation data matched with each WSI patient were collected and processed by median bins. Using the processing pipeline, the top 10 genes were determined - EPP, PRICKLE4, ANK3, CABLES2, ZSCAN2, KDM4B, ZNF556, ZNRF1, MMEL1 and PLGLB2, with AUCs ranging from 0.855 to 0.903.

[0033] The same method was used to predict the gene mutation status of bladder cancer using deep learning. In a specific embodiment, in order to ensure that there were enough mutation samples in the training, we only used genes with a mutation rate of more than 10%, thus reducing the number of genes to be analyzed to an acceptable range; we directly trained a model from scratch for each selected gene using ResNet-50 on a 400-fold enlarged dataset; the top five mutation genes predicted by deep learning were COL11A1, FGFR3, SRRM2, LRP1B and MUC17, with AUC values ​​ranging from 0.664 to 0.775.

[0034] In one embodiment, obtaining training samples includes: obtaining training WSIs of several HE-stained bladder tissues in the public TCGA; labeling the training WSIs, and filtering the training WSIs to obtain several WSI images that only retain labels of gene expression, gene methylation, and gene mutation; dividing the several WSI images to obtain a verification WSI set, a training WSI set, and a test WSI set; based on a 400-fold magnification, dividing all WSI sets into image blocks of 320×320 pixels, and filtering the background to obtain a second verification WSI set, a second training WSI set, and a second test WSI set; wherein all WSI sets include the verification WSI set, the training WSI set, and the test WSI set;

[0035] The working principle and beneficial effects of the above technical scheme are as follows: we used the publicly available WSI of H&E-stained bladder tissue in TCGA (Cancer Genome Atlas) to develop and evaluate our model; we filtered out the WSI with poor quality and only retained the images with gene expression, gene methylation and gene mutation labels; after screening, we obtained 400 WSIs, of which 250 were used for training, 50 were used for validation, and 103 were used as test sets; since the size of WSI was too large to be directly used as the input of the neural network, we divided the WSI into image blocks of 320×320 pixels at 400 times magnification and filtered out the background; finally, we obtained a training set containing 150,000 image blocks (the second training WSI set), a validation set containing 50,000 image blocks (the second validation WSI set), and a test set containing 100,000 image blocks (the second test WSI set).

[0036] In one embodiment, a bladder cancer gene mutation prediction system based on deep learning also includes a two-stage pre-analysis method to accelerate the training process. The specific acceleration process is: in the first stage, based on a 100-fold magnification, all WSI sets are divided into image blocks of 320×320 pixels, and the background is filtered to obtain a third verification WSI set, a third training WSI set, and a third test WSI set for constructing an analysis data set, wherein all WSI sets include a verification WSI set, a training WSI set, and a test WSI set; the analysis data set is converted into a RestNet-50 model pre-trained on ImageNet. Each image block in the image is converted into a 2048-dimensional vector as a feature extractor; in the second stage, the 3-fold cross-validation method in the k-fold cross-validation method is used to evaluate the performance of Xgboost trained on the feature vector to obtain the evaluation results; wherein, in each fold, 200 WSIs in the analysis data set are randomly selected for training, and 100 WSIs are selected for testing; based on the RestNet-50 model, a classification model is rebuilt on the second validation WSI set, the second training WSI set, and the second test WSI set, and the evaluation results are combined to provide detailed classification information for each selected gene for labeling;

[0037] The working principle and beneficial effects of the above technical scheme are as follows: In order to improve the analysis efficiency, we proposed a two-stage pre-analysis method to speed up the process; in the first stage, we constructed a smaller dataset at 100 times magnification using 320×320 pixel image blocks in a similar way; then, each image block was converted into a 2048-dimensional vector as a feature extractor using the RestNet-50 model pre-trained on ImageNet; these feature vectors were subsequently used as input to the next stage, with the advantage of fast analysis; in the second stage, we evaluated the performance of Xgboost trained on the feature vectors using 3-fold cross validation; in each fold, 200 WSIs were randomly selected for training, while 100 WSIs were selected for testing to ensure that there were no overlapping WSIs between datasets; the first few genes selected by the two-stage pre-analysis method were considered predictive; finally, we built a classification model from scratch on the 400-fold magnified dataset using RestNet-50 to provide more detailed information for each selected gene; based on this framework, we provide results for three gene-related prediction tasks, including gene expression, gene methylation, and gene mutation status.

[0038] In one embodiment, the report generation module performs the following operations: obtaining the corresponding prediction result, extracting the corresponding original case data from the preset database according to the corresponding prediction result, wherein the original case data is the case data of the patient corresponding to the current prediction result participating in the prediction during this prediction; extracting the valid initial text corpus in the original case data based on the regular expression, wherein each initial text corpus uniquely corresponds to a prediction result; sorting all the initial text corpora according to the text length, counting the text length, and constructing a text length relationship table; performing entity extraction of the medical entity type on each initial text corpus, and determining the entity of each initial text corpus. The entity type of the corpus is associated with the entity types of all the initial text corpora under each prediction result according to the preset entity association relationship to obtain the entity association relationship of each initial text corpus; the initial report template is determined according to the longest text length in the text length relationship table, and based on the entity association relationship of the initial text corpus under each prediction result, any number of associated initial text corpora are merged and completed to obtain the report text to be generated; based on the initial report text filling logic corresponding to the initial report template, the report text to be generated and the prediction result are matched and filled into the initial report template to generate a bladder cancer gene mutation prediction auxiliary diagnosis report;

[0039] The working principle of the above technical solution is: obtaining the prediction results of bladder cancer gene expression status, bladder cancer gene methylation status and bladder cancer gene mutation status, extracting the corresponding original case data from the preset database based on the above multiple prediction results, wherein the original case data is the case data of the patient corresponding to the current prediction result participating in the prediction at this time, and preferably, the case data is attached with standard data for comparison; then extracting the valid initial text corpus from the case data participating in the prediction at this time through regular expressions, wherein each initial text corpus uniquely corresponds to a prediction result; then sorting the initial text corpus according to the text length, and counting the text length to obtain the sorting result and the counting result, and constructing a text length relationship table; performing entity extraction of the medical entity type on each initial text corpus, and determining the medical entity type of each initial text corpus, the medical entity type includes multiple, such as diseases, clinical manifestations, drugs, medical equipment, medical procedures, body, medical examination Test items, microbiology, departments, etc., according to the preset entity association relationship, the entity types of all initial text corpora under each prediction result are associated to obtain the entity association relationship of each initial text corpus; the initial report template is determined according to the longest text length in the text length relationship table, and based on the entity association relationship of the initial text corpus under each prediction result, any number of associated initial text corpora are merged and completed to obtain the report text to be generated; wherein, any number is at least 2, and the entity types corresponding to more than 2 initial text corpora must be associated with each other, for example, there are three entity types A, B, and C, then A and B must be able to form an entity pair, A and C must be able to form an entity pair, and B and C must be able to form an entity pair; and the text length of the merged text corpus after merging and completing is less than the longest text length; based on the initial report text filling logic corresponding to the initial report template, the report text to be generated and the prediction result are matched and filled into the initial report template to generate a bladder cancer gene mutation prediction auxiliary diagnosis report;

[0040] The beneficial effect of the above technical solution is: through the above technical solution, the auxiliary diagnosis report is generated by means of entity association and entity extraction, which is beneficial to improve the report generation speed, and at the same time, through the entity association method, it speeds up the user's understanding of the report.

[0041] See also Figure 2 , Figure 3In one embodiment, a bladder cancer gene mutation prediction system based on deep learning also includes: S101, obtaining the user's diagnostic intention uploaded by the designated terminal, the user's diagnostic intention includes various medical subjects, wherein the user's diagnostic intention includes multiple; S102, analyzing all the user's diagnostic intentions uploaded this time, and determining the diagnostic field that the user needs auxiliary diagnosis this time; S103, removing irrelevant content in the bladder cancer gene mutation prediction auxiliary diagnosis report based on the diagnostic field, generating a corresponding accurate auxiliary diagnosis report and feeding it back to the designated terminal; wherein, analyzing all the user's diagnostic intentions uploaded this time, and determining the diagnostic field that the user needs auxiliary diagnosis this time The domain includes: S201, establishing a diagnostic intention database, wherein the diagnostic intention database stores a number of diagnostic intentions that have been marked with diagnostic fields, each diagnostic intention is marked with an association value with different diagnostic fields, and each diagnostic intention is related to several diagnostic fields at the same time; S202, based on the diagnostic intention database, generating a first association value m between each user's diagnostic intention and different diagnostic fields, wherein each user's diagnostic intention has a first association value m with each diagnostic field; S203, based on all the first association values ​​m, counting a number of second association values ​​of the user's diagnostic intention corresponding to any diagnostic field; the second association values ​​include m1, m2...m n , where n is the total number of user diagnostic intentions; S204, screen out all user diagnostic intentions corresponding to the second correlation values ​​exceeding the preset statistical threshold p as target diagnostic intentions, and calculate the number k of target diagnostic intentions in each diagnostic field i , where i is the number of diagnostic fields; S205, counting the number k of target diagnostic intentions in each diagnostic field i The proportion of the total number of users' diagnostic intentions i ; S206, pre-set condition to meet the value g, when k i With j i When the product of is not less than g, take k i With j i The diagnostic field corresponding to the maximum value of the product of is output as the target diagnostic field, and the diagnostic field that the user needs auxiliary diagnosis for this time is determined;

[0042] The working principle of the above technical solution is: obtaining the user's diagnostic intention uploaded by the designated terminal, the user's diagnostic intention includes various medical subjects, for example, surgery, internal medicine, etc. in the first-level subjects, respiratory medicine, gastroenterology, etc. in the second-level subjects; wherein the user's diagnostic intention can include multiple; analyzing all the user's diagnostic intentions uploaded this time, determining the diagnostic field that the user needs auxiliary diagnosis this time, and removing irrelevant content in the bladder cancer gene mutation prediction auxiliary diagnosis report based on the diagnostic field, generating a corresponding accurate auxiliary diagnosis report and feeding it back to the designated terminal; wherein, analyzing all the user's diagnostic intentions uploaded this time, and determining the diagnostic field that the user needs auxiliary diagnosis this time includes: establishing a diagnostic intention database, wherein the diagnostic intention database stores a number of diagnostic intentions with marked diagnostic fields, each diagnostic intention is marked with an association value with different diagnostic fields, and each diagnostic intention can be simultaneously associated with a number of The correlation value is output and marked in advance according to the correlation value scoring model; a first correlation value m is generated for each user's diagnostic intention and different diagnostic fields, wherein each user's diagnostic intention has a first correlation value m with each different diagnostic field. If there is a user's diagnostic intention that is not associated with the current diagnostic field, its first correlation value is 0; for example, if there are 10 user diagnostic intentions and 10 diagnostic fields in the database, the number of first correlation values ​​m is 100, and 10 first correlation values ​​will be generated in each diagnostic field; or, if there are 9 user diagnostic intentions and 10 diagnostic fields in the database, the number of first correlation values ​​m is 90, and 9 first correlation values ​​will be generated in each diagnostic field; based on all the first correlation values ​​m, a number of second correlation values ​​of the user's diagnostic intention corresponding to any diagnostic field are counted, and the second correlation values ​​include m1, m2...m n , where n is the total number of user diagnostic intentions, and then all user diagnostic intentions corresponding to the second correlation values ​​exceeding the preset statistical threshold p are screened as target diagnostic intentions, and the number of target diagnostic intentions in each diagnostic field is calculated. i , where i is the number of diagnostic fields; the number of target diagnostic intentions in each diagnostic field is counted k i The proportion of the total number of users' diagnostic intentions i , i.e. j i =k i / n; preset condition meets value g, when k i With j i When the product of is not less than g, take k i With j i The diagnostic field corresponding to the maximum value of the product of is output as the target diagnostic field; it is worth noting that when k i With j iWhen the product of is less than g, the output association error is fed back to the designated terminal, and the bladder cancer gene mutation prediction auxiliary diagnosis report is output to the instruction terminal at the same time; wherein, the evaluation process of the association value of each user's diagnostic intention and different diagnostic fields is preferably performed by extracting the diagnosis text feature from each user's diagnostic intention, and the diagnosis text feature includes but is not limited to feature information such as text length and diagnosis entity naming, using a deep learning model to establish an association value scoring model, and annotating each diagnosis text feature in advance through a high-accuracy labeling system, and the annotated data includes different diagnostic field data, and then optimizing and verifying the association value scoring model through the annotated feature data to improve the association value scoring model; in actual use, the user's diagnostic intention is input into the association value scoring model, and the association value of each user's diagnostic intention and different diagnostic fields is output;

[0043] The beneficial effect of the above technical solution is: through the above technical solution, the user's diagnostic intention is analyzed, and the diagnostic intention is matched with the actual diagnostic field, so as to streamline the content of the auxiliary report, so that useless data will not appear in the auxiliary report, and personalized services can be provided to users. At the same time, the streamlined auxiliary report can reduce the probability of user distraction, so that the user will not be distracted by irrelevant data.

[0044] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A bladder cancer gene mutation prediction system based on deep learning, characterized in that: include: A histopathology image acquisition module, used for acquiring original histopathology images to be tested; The preprocessing module is used to preprocess the original tissue pathology image to be tested using HE staining and a whole slide imaging scanner to obtain WSI with HE-stained bladder tissue; the cloud server module is used to call the bladder cancer deep learning prediction model to predict the bladder cancer gene expression status, bladder cancer gene methylation status and bladder cancer gene mutation status of the WSI and generate corresponding prediction results; the report generation module is used to automatically generate a bladder cancer gene mutation prediction auxiliary diagnosis report based on the corresponding prediction results, and feed it back to the designated terminal.

2. A bladder cancer gene mutation prediction system based on deep learning according to claim 1, characterized in that: The histopathology image acquisition module is connected to a preset urinary system image acquisition device and is used to acquire the original histopathology image to be tested acquired by the preset urinary system image acquisition device.

3. A bladder cancer gene mutation prediction system based on deep learning according to claim 1, characterized in that: The preprocessing module performs the following operations: staining the bladder tissue to be tested according to the HE staining method to obtain the HE-stained bladder tissue; and using a whole slide imaging scanner to obtain the WSI of the HE-stained bladder tissue according to the original pathological image of the tissue to be tested after being stained by the HE staining method.

4. A bladder cancer gene mutation prediction system based on deep learning according to claim 1, characterized in that: The training process of the deep learning prediction model for bladder cancer includes the following steps: obtaining training samples; constructing a deep learning framework based on a convolutional neural network, training the deep learning framework according to the training samples until the preset conditions are met and the training is terminated to generate a deep learning prediction model for bladder cancer.

5. A bladder cancer gene mutation prediction system based on deep learning according to claim 4, characterized in that: Acquiring training samples includes: acquiring several training WSIs of HE-stained bladder tissues from the public TCGA; labeling the training WSIs, and filtering the training WSIs to obtain several WSI images that only retain labels of gene expression, gene methylation, and gene mutation; dividing the several WSI images to obtain a verification WSI set, a training WSI set, and a test WSI set; based on a 400-fold magnification, dividing all WSI sets into image blocks of 320×320 pixels, and filtering the background to obtain a second verification WSI set, a second training WSI set, and a second test WSI set; wherein all WSI sets include a verification WSI set, a training WSI set, and a test WSI set.

6. A bladder cancer gene mutation prediction system based on deep learning according to claim 5, characterized in that: It also includes a two-stage pre-analysis method to accelerate the training process. The first stage acceleration process includes: based on a 100-fold magnification, all WSI sets are divided into image blocks of 320×320 pixels, and the background is filtered to obtain a third verification WSI set, a third training WSI set, and a third test WSI set for constructing an analysis data set; the RestNet-50 model pre-trained on ImageNet is used to convert each image block in the analysis data set into a 2048-dimensional vector as a feature extractor; wherein all WSI sets include verification WSI sets, training WSI sets, and test WSI sets.

7. A bladder cancer gene mutation prediction system based on deep learning according to claim 6, characterized in that: The two-stage acceleration process includes: using the 3-fold cross-validation method in the k-fold cross-validation method to evaluate the performance of Xgboost trained on the feature vector to obtain the evaluation results; in each fold, 200 WSIs in the analysis data set are randomly selected for training, and 100 WSIs are selected for testing; based on the RestNet-50 model, a classification model is rebuilt on the second validation WSI set, the second training WSI set, and the second test WSI set, and the evaluation results are combined to provide detailed classification information for each selected gene for labeling.

8. The deep learning-based bladder cancer gene mutation prediction system according to claim 1, characterized in that: The report generation module performs the following operations: obtaining the corresponding prediction results, extracting the corresponding original case data from the preset database according to the corresponding prediction results, wherein the original case data is the case data of the patient corresponding to the current prediction result participating in the prediction during this prediction; extracting the valid initial text corpus in the original case data based on regular expressions, wherein each initial text corpus uniquely corresponds to a prediction result; sorting all the initial text corpora according to the text length, counting the text length, and constructing a text length relationship table; performing entity extraction of the medical entity type for each initial text corpus, and determining the Entity type, according to the preset entity association relationship, associate the entity types of all initial text corpora under each prediction result to obtain the entity association relationship of each initial text corpus; determine the initial report template according to the longest text length in the text length relationship table, and based on the entity association relationship of the initial text corpus under each prediction result, merge and complete any number of associated initial text corpora to obtain the report text to be generated; based on the initial report text filling logic corresponding to the initial report template, fill the report text to be generated and the prediction result into the initial report template to generate a bladder cancer gene mutation prediction auxiliary diagnosis report.

9. The deep learning-based bladder cancer gene mutation prediction system according to claim 1, characterized in that: Also includes: The user diagnostic intent uploaded by the designated terminal is obtained, and the user diagnostic intent includes various medical subjects, among which the user diagnostic intent includes multiple ones; all the user diagnostic intents uploaded this time are analyzed to determine the diagnostic field that the user needs auxiliary diagnosis for this time, and based on the diagnostic field, irrelevant content in the auxiliary diagnosis report for bladder cancer gene mutation prediction is removed, and the corresponding accurate auxiliary diagnosis report is generated and fed back to the designated terminal.

10. A bladder cancer gene mutation prediction system based on deep learning according to claim 9, characterized in that: Analyze all the user diagnostic intentions uploaded this time to determine the diagnostic field that the user needs auxiliary diagnosis for this time, including: establish a diagnostic intention database, wherein the diagnostic intention database stores several diagnostic intentions with marked diagnostic fields, each diagnostic intention is marked with an association value with different diagnostic fields, and each diagnostic intention is related to several diagnostic fields at the same time; based on the diagnostic intention database, generate a first association value m between each user diagnostic intention and the diagnostic field, wherein each user diagnostic intention has a first association value m with each diagnostic field; based on all the first association values ​​m, count several second association values ​​of the user diagnostic intentions involved in any diagnostic field, and the second association values ​​include m1, m2...m n , where n is the total number of user diagnostic intentions; all user diagnostic intentions corresponding to the second correlation values ​​exceeding the preset statistical threshold p are screened as target diagnostic intentions, and the number of target diagnostic intentions in each diagnostic field is calculated. i , where i is the number of diagnostic fields; the number of target diagnostic intentions in each diagnostic field is counted k i The proportion of the total number of users' diagnostic intentions i ; Pre-set condition to meet value g, when k i With j i When the product of is not less than g, take k i With j i The diagnostic field corresponding to the maximum value of the product is output as the target diagnostic field to determine the diagnostic field that the user needs auxiliary diagnosis for this time.

Images