Cancer treatment effect prediction method and device, equipment and storage medium

Abstract

The invention relates to the technical field of machine learning, and discloses a cancer treatment effect prediction method and device, equipment and a storage medium. The method comprises the following steps: carrying out radiomics feature extraction on a CT scanning image to obtain radiomics features; performing interested pathological feature extraction based on an attention mechanism and local pathological feature extraction based on an interested pathological feature extraction result on the pathological image to obtain pathological omics features; performing differential expression gene analysis on the transcriptomics data, and performing differential expression gene screening based on a differential expression gene analysis result to obtain transcriptomics characteristics; and inputting a fusion feature obtained by fusing the radiomics feature, the pathological omics feature, the transcriptomics feature and the clinical data into a cancer treatment effect prediction model to obtain a cancer treatment effect prediction result. According to the embodiment of the invention, the cancer treatment effect is predicted by combining the radiomics characteristics, the pathomics characteristics, the transcriptomics characteristics and the clinical characteristics, and the prediction accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a method, device, equipment and storage medium for predicting the efficacy of cancer treatment. Background Art

[0002] Cancer, also known as malignant tumor, is caused by malignant cell proliferation and is invasive and metastatic.

[0003] In related technologies, the main treatment methods for cancer are radiotherapy, chemotherapy and surgery. Machine learning-based imaging omics has a good application in identifying cancer types, predicting treatment effects and predicting disease progression. However, the current machine learning-based imaging omics-based cancer treatment efficacy prediction method is based on a single modality feature and has low prediction accuracy. Summary of the invention

[0004] The purpose of this application is to provide a method, device, equipment and storage medium for predicting the efficacy of cancer treatment. By combining imaging genomics features, pathological genomics features, transcriptomics features and clinical characteristics to predict the efficacy of cancer treatment, the prediction accuracy can be improved.

[0005] The present application embodiment provides a method for predicting the efficacy of cancer treatment, comprising: Obtain CT scan images, pathological images, transcriptomic data, and clinical data; Extracting radiomics features from the CT scan image to obtain radiomics features; Extracting pathological features of interest based on an attention mechanism and extracting local pathological features based on the results of the pathological features of interest from the pathological image to obtain pathological omics features; Performing differentially expressed gene analysis on the transcriptomic data and screening differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features; fusing the imaging omics features, the pathological omics features, the transcriptomics features, and the clinical data to obtain a fusion feature; The fusion feature is input into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion feature to obtain a cancer treatment efficacy prediction result.

[0006] In some embodiments, the extracting radiomics features from the CT scan image to obtain radiomics features includes: Segmenting the CT scan image into regions of interest to obtain a plurality of CT scan image blocks; Performing radiomics feature extraction on the CT scan image block to obtain the radiomics feature; the radiomics feature extraction includes extracting first-order statistical features, regional shape and size features, and texture features in the CT scan image block.

[0007] In some embodiments, the pathological image is subjected to the extraction of pathological features of interest based on an attention mechanism and the extraction of local pathological features based on the pathological features of interest extraction results to obtain pathological omics features, including: Segmenting the pathological image to obtain a plurality of pathological image blocks; Based on the attention mechanism, the pathological features of interest in the pathological image blocks are sensed to select a number of the pathological image blocks with the highest attention scores to obtain a number of candidate image blocks; the pathological features of interest are pathological features that characterize no tumor residue after cancer treatment; Local pathological features are extracted from the candidate image blocks to obtain the pathological omics features.

[0008] In some embodiments, the attention mechanism is used to perceive the pathological features of interest in the pathological image blocks to select several pathological image blocks with the highest attention scores to obtain several candidate image blocks, including: Performing encoding processing on the pathological image block to obtain an encoding output vector; Performing full-connection dimensionality reduction processing on the encoded output vector to obtain a full-connection output vector; According to the importance of the pathological features of interest in the pathological image block, an attention weight operation is performed on the fully connected output vector to obtain a number of attention scores, and a number of the pathological image blocks with the highest attention evaluation scores are selected to obtain a number of candidate image blocks.

[0009] In some embodiments, the performing differentially expressed gene analysis on the transcriptomic data and screening differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features includes: Comparing the transcriptomic data with reference genome data to obtain a gene expression matrix corresponding to the transcriptomic data; Performing differential expression analysis on the gene expression matrix for cancer treatment efficacy at the gene level, and screening out a number of differentially expressed genes according to the differential expression analysis results; Linear regression analysis is performed on the gene expression values ​​corresponding to the differentially expressed genes to screen out the differentially expressed genes that meet the linear regression analysis conditions and obtain the transcriptomic features.

[0010] In some embodiments, the step of inputting the fusion feature into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion feature to obtain a cancer treatment efficacy prediction result includes: Using a preset kernel function, in the cancer treatment efficacy prediction model, the fusion feature is mapped to a high-dimensional space for linear transformation to obtain a high-dimensional mapping vector of the fusion feature; Calculate the loss value for each treatment efficacy category based on a preset classification decision function; The high-dimensional mapping vector is classified and predicted based on the loss value of the treatment efficacy category to obtain the cancer treatment efficacy prediction result.

[0011] In some embodiments, the method for predicting the efficacy of cancer treatment further comprises: According to the cancer treatment efficacy prediction result, the patient's survival probability is calculated and a significance analysis is performed on the survival probability to obtain a survival analysis result.

[0012] The present application also provides a cancer treatment efficacy prediction device, comprising: The first module is used to obtain CT scan images, pathological images, transcriptomic data and clinical data; The second module is used to extract radiomics features from the CT scan image to obtain radiomics features; The third module is used to extract the pathological features of interest based on the attention mechanism and extract the local pathological features based on the results of the pathological features of interest to obtain the pathological omics features; The fourth module is used to perform differentially expressed gene analysis on the transcriptomic data and screen differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features; A fifth module is used to fuse the radiomics feature, the pathological features, the transcriptomics feature and the clinical data to obtain a fusion feature; The sixth module is used to input the fusion features into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion features to obtain a cancer treatment efficacy prediction result.

[0013] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned cancer treatment efficacy prediction method when executing the computer program.

[0014] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for predicting the efficacy of cancer treatment is implemented.

[0015] The beneficial effects of the present application are as follows: CT scan images, pathological images, transcriptomic data and clinical data of patients are obtained, and corresponding features are extracted to obtain imaging features, pathological features, transcriptomic features and clinical data, and the fusion features obtained by fusing imaging features, pathological features, transcriptomic features and clinical data are input into a cancer treatment efficacy prediction model, so as to evaluate the cancer treatment efficacy based on the fusion features and obtain the cancer treatment efficacy prediction results. Since imaging features, pathological features, transcriptomic features and clinical features are combined, the cancer treatment efficacy prediction is performed based on multiple modal information, which can improve the accuracy of cancer treatment efficacy prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flow chart of the method for predicting the efficacy of cancer treatment provided in the embodiment of the present application.

[0017] Figure 2 It is a flowchart of the specific method of step S103 provided in an embodiment of the present application.

[0018] Figure 3 It is a flowchart of the specific method of step S202 provided in an embodiment of the present application.

[0019] Figure 4 It is a flowchart of the specific method of step S104 provided in an embodiment of the present application.

[0020] Figure 5 It is a flowchart of the specific method of step S106 provided in an embodiment of the present application.

[0021] Figure 6 It is a schematic diagram of the structure of a cancer treatment efficacy prediction device provided in an embodiment of the present application.

[0022] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown can be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second" and the like in the specification, claims and drawings are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0026] See also Figure 1 , Figure 1 is a flow chart of a method for predicting the efficacy of cancer treatment provided in an embodiment of the present application. In some embodiments, Figure 1 The method includes but is not limited to steps S101 to S106.

[0027] It should be noted that the execution subject of the cancer treatment efficacy prediction method provided in the embodiments of this specification can be applied to terminal-side devices or cloud-side devices, and this embodiment does not limit this.

[0028] Step S101, acquiring CT scan images, pathological images, transcriptomics data and clinical data.

[0029] The executing entity can obtain CT scan images, pathological images, transcriptomic data and clinical data. CT scan images, pathological images, transcriptomic data and clinical data are all obtained by testing patients with the same cancer disease through corresponding means.

[0030] In some embodiments, the cancer disease may be esophageal squamous cell carcinoma.

[0031] CT scan images are generated by using X-rays to scan the human body and processing the scan data with a computer. CT scan images are cross-sectional images that can show the tissue density distribution of a certain cross-sectional area of ​​the human body. They have high density resolution and can clearly show soft tissue and bone structures.

[0032] Pathological images are images formed by scanning and fusion of pathological tissues after dehydration, embedding, and slicing, etc., and then scanning and fusion by a microscopic scanning device. The production of pathological images first involves scanning the microscopic tissue images of pathological slices with a scanning device, and then fusion of the captured images. During the shooting process, there are overlapping parts of adjacent images, and the fusion area is determined through operations such as positioning and image registration to obtain a pathological image.

[0033] Transcriptomics data is gene expression data obtained through transcriptome sequencing technology. Transcriptomics is a discipline that studies gene transcription in cells and its regulatory rules, mainly studying gene expression at the RNA level. The transcriptome is the sum of all RNAs that can be transcribed by a living cell, including mRNA, tRNA, rRNA, and various non-coding RNAs.

[0034] Clinical data is data obtained from medical institutions that is directly related to doctors' work with patients and diseases. These data involve safety and performance information during the clinical use of medical devices, and may come from pre-market and post-market clinical trial information, relevant research materials, and clinical experience materials. In some embodiments, clinical data include clinical information such as the patient's age, gender, weight, lesion length, number of CT lymph nodes, clinical stage, and pathological grade.

[0035] Step S102 , extracting radiomics features from the CT scan image to obtain radiomics features.

[0036] The execution subject can extract radiomics features from the CT scan image to obtain radiomics features.

[0037] In a specific implementation, the executing entity performs radiomics feature extraction on the CT scan image. Radiomics feature extraction refers to extracting representative features from the CT scan image. These features can describe the morphology, structure, texture and other information of the area of ​​interest related to the disease state in the CT scan image.

[0038] In some embodiments, the above step S102 specifically includes: segmenting the CT scan image into regions of interest to obtain a number of CT scan image blocks; and extracting radiomics features from the CT scan image blocks to obtain radiomics features. The radiomics feature extraction includes extracting first-order statistical features, regional shape and size features, and texture features from the CT scan image blocks.

[0039] In a specific implementation, the region of interest segmentation of the CT scan image can be performed by three-dimensional manual segmentation based on 3D-Slicer software, and an oncologist with more than 5 years of experience outlines the region of interest for each layer of the CT scan image, and the region of interest contains the tumor. The CT scan image is then segmented according to the segmentation result to obtain a number of CT scan image blocks containing the region of interest, and then the CT scan image blocks are subjected to radiomics feature extraction, including extracting first-order statistical features, regional shape and size features, and texture features in the CT scan image blocks. The texture features can be 100 radiomics features including gray-level co-occurrence matrix (GLCM), gray-level dependence matrix (GLDM), gray-level free length matrix (GLRLM), gray-level size zone matrix (GLSZM), etc. Finally, the extracted features are fitted to obtain the radiomics features.

[0040] Step S103, extracting pathological features of interest based on the attention mechanism and extracting local pathological features based on the results of the pathological features of interest from the pathological image to obtain pathological omics features.

[0041] The execution subject can extract pathological features of interest based on the attention mechanism and extract local pathological features based on the results of pathological feature extraction to obtain pathological genomics features.

[0042] In the specific implementation, the executing entity first extracts the pathological features of interest based on the attention mechanism for the pathological image. The pathological features of interest are pathological features that characterize the absence of tumor residues after cancer treatment. Based on the attention mechanism, several local image regions in the pathological image containing the pathological features of interest with the highest attention evaluation scores are selected as the results of the pathological features of interest extraction. Then, local pathological feature extraction is performed on the local image regions to extract multiple different types of local pathological features in the local image regions. Finally, the extracted local pathological features are fitted to obtain the pathological omics features.

[0043] Figure 2 is a flowchart of a specific method of step S103 provided in an embodiment of the present application. Figure 2 In some embodiments, the method includes but is not limited to steps S201 to S203.

[0044] Step S201, segmenting the pathological image to obtain a plurality of pathological image blocks.

[0045] In some embodiments, the pathological image is segmented by maximizing the inter-class variance between the foreground and background of the pathological image based on an adaptive threshold selection algorithm to find the best threshold, and then segment the pathological image into image blocks of a preset size to obtain multiple pathological image blocks. For example, the tissue area of ​​each pathological image is automatically segmented using the Ostu threshold, read into the memory at a 20x magnification, and the pathological image is segmented into multiple pathological image blocks of 256×256 pixels.

[0046] Step S202: Based on the attention mechanism, the pathological features of interest in the pathological image blocks are perceived to select several pathological image blocks with the highest attention scores to obtain several candidate image blocks.

[0047] Among them, the pathological features of interest are pathological features that characterize the absence of tumor residue after cancer treatment.

[0048] In some embodiments, the pathological image block may be input into a pre-trained pathological feature extraction model of interest, and the pathological feature extraction model of interest may perceive the pathological feature of interest in the pathological image block based on the attention mechanism, evaluate the importance of the perceived pathological feature of interest and assign an attention score, output several pathological image blocks with the highest attention scores, and obtain several candidate image blocks.

[0049] Figure 3 is a flowchart of a specific method of step S202 provided in an embodiment of the present application. Figure 3 In some embodiments, the pathological feature extraction model of interest includes an encoder layer, a fully connected layer, and an attention perception layer. The method includes but is not limited to steps S301 to S303.

[0050] Step S301, encoding the pathological image block to obtain an encoding output vector.

[0051] Step S302, performing full connection dimensionality reduction processing on the encoded output vector to obtain a full connection output vector.

[0052] Step S303, according to the importance of the pathological features of interest in the pathological image blocks, an attention weight operation is performed on the fully connected output vector to obtain a number of attention scores, and a number of pathological image blocks with the highest attention scores are selected to obtain a number of candidate image blocks.

[0053] In the encoder layer, the pathological image block is encoded to obtain an encoded output vector. In the specific implementation, the encoder layer is mainly composed of a pre-trained ResNet50, which is trained by clustering-guided contrastive learning. The pathological image block is input into the encoder layer, and the pathological image block is encoded to obtain an encoded output vector.

[0054] In the fully connected layer, the encoded output vector is subjected to a fully connected dimension reduction process to obtain a fully connected output vector. In a specific implementation, a convolutional network containing 512 neurons is used as the main body to form a fully connected layer, and the encoded output vector is input into the fully connected layer to perform a fully connected dimension reduction process on the encoded output vector to obtain a fully connected output vector.

[0055] In the attention perception layer, according to the importance of the pathological features of interest in the pathological image blocks, the attention weight operation is performed on the fully connected output vector to obtain several attention scores, and several pathological image blocks with the highest attention scores are selected to obtain several candidate image blocks. In the specific implementation, two attention networks are stacked to form the attention perception layer. The attention network is composed of three fully connected networks stacked. The two attention networks are used together as part of the attention trunk shared by two categories. Then the first two layers of attention networks are divided into two parallel attention branches, and these parallel independent classifiers are constructed to score the slide-level representation specific to each category to obtain the corresponding attention score. After obtaining the attention score, the pathological image blocks are sorted according to the attention score, and several pathological image blocks with the highest attention scores are selected as candidate image blocks.

[0056] The calculation formula of the attention weight score is: , in, is the attention weight score, , and are the attention weights, and are all elements in the fully connected output vector, N is the number of pathological image blocks, is the Hadamard product, which means element-by-element multiplication, tanh() is the hyperbolic tangent function, and sigm() is the sigmoid activation operation.

[0057] Step S203: extract local pathological features from the candidate image blocks to obtain pathological genomic features.

[0058] In some embodiments, local pathological features are extracted from candidate image blocks by using a pre-trained MaskRCNN model to extract local pathological features from each candidate image block. For each candidate image block, local pathological features such as area, convex area, eccentricity, range, filling area, major axis length, minor axis length, direction, perimeter, firmness, tumor cell ratio, lymphocyte ratio, stromal cell ratio, macrophage ratio, nuclear rupture ratio and red blood cell ratio are extracted. Finally, each local pathological feature of each candidate image block is averaged, and the local pathological features obtained by the average operations are fitted to obtain the pathological omics features.

[0059] Step S104, performing differentially expressed gene analysis on the transcriptomic data and screening differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features.

[0060] The executing entity can perform differentially expressed gene analysis on the transcriptomic data and screen differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features.

[0061] In the specific implementation, the executing entity first preprocesses the transcriptomics data to remove low-quality gene expression values, and then performs differentially expressed gene analysis on the preprocessed transcriptomics data to select differentially expressed genes for cancer treatment efficacy at the genetic level, and then uses the linear regression method to learn the gene expression values ​​corresponding to the differentially expressed genes, and further screens the selected differentially expressed genes to screen out the corresponding differentially expressed genes and obtain transcriptomics features.

[0062] Figure 4 is a flowchart of a specific method of step S104 provided in an embodiment of the present application. Figure 4 In some embodiments, the method includes but is not limited to steps S401 to S403.

[0063] Step S401 : Compare the transcriptomics data with the reference genome data to obtain a gene expression matrix corresponding to the transcriptomics data.

[0064] In some embodiments, an appropriate method is selected to detect sample quality based on the characteristics of the biological sample, and mRNA enrichment, reverse transcription, library construction and sequencing are performed on qualified samples to obtain raw sequencing data. The raw sequencing data is subjected to FastQC quality control to check the sequencing data quality, and then high-quality transcriptomics data is obtained after removing linkers, low-quality sequences and contamination.

[0065] In some embodiments, the transcriptomics data and the reference genome data are compared by aligning the transcriptomics data with the reference genome based on the STAR tool to obtain a gene expression matrix of the transcriptomics data.

[0066] Step S402, performing differential expression analysis on the gene expression matrix for cancer treatment efficacy at the gene level, and screening out a number of differentially expressed genes according to the differential expression analysis results.

[0067] In some embodiments, a differential expression analysis of the efficacy of cancer treatment at the gene level is performed on the gene expression matrix, and a number of differentially expressed genes are screened out according to the results of the differential expression analysis. The gene expression matrix can be compared with a reference gene expression matrix that is divided into a PCR group (no tumor residue after cancer treatment) and a reference gene expression matrix that is divided into a non-PCR group (tumor residue after cancer treatment), and the differentially expressed genes in the gene expression matrix are screened out by setting corresponding threshold conditions. For example, a threshold condition of P < 0.05 and |log(2)FC| ≥ 1 is set (P is the expression difference threshold, FC is the difference multiple), and 791 differentially expressed genes are screened out from the gene expression matrix, of which 501 are up-regulated genes and 290 are down-regulated genes.

[0068] Step S403 , performing linear regression analysis on the gene expression values ​​corresponding to the differentially expressed genes to screen out the differentially expressed genes that meet the linear regression analysis conditions and obtain transcriptomic features.

[0069] In some embodiments, linear regression analysis is performed on the gene expression values ​​corresponding to the differentially expressed genes, and LASSO (Least absolute shrinkage and selection operator) regression can be used to screen out more refined differentially expressed genes, so as to screen out differentially expressed genes that meet the conditions of linear regression analysis and obtain transcriptomic features. LASSO regression is a linear regression method that uses L1 regularization. The use of L1 regularization will make the feature weights of some learned unimportant differentially expressed genes 0, and screen out differentially expressed genes whose feature weights are not 0 after LASSO regression, thereby achieving the purpose of sparsification and feature selection. Among them, the expression of LASSO regression is: , in, is the objective function value of LASSO regression, n is the number of differentially expressed genes, is the true value of the ith differentially expressed gene, is the characteristic value of the ith differentially expressed gene, is the regression coefficient of the jth differentially expressed gene, is the regularization strength, and p is the number of features of differentially expressed genes.

[0070] Step S105, integrating radiomics features, pathological features, transcriptomics features and clinical data to obtain fusion features.

[0071] The execution entity can fuse radiomics features, pathological features, transcriptomics features and clinical data to obtain fusion features.

[0072] In the specific implementation, the execution subject vectorizes the radiomics features, pathological features, transcriptomics features and clinical data and then splices them to obtain the fusion features in the form of input vectors that meet the cancer treatment efficacy prediction model. The expression for obtaining the fusion features is: , in, is the vector representation of the fusion feature, is the vector representation of radiomics features, is the vector representation of pathological omics features, is the vector representation of transcriptomic features, is the vector representation of clinical data, For splicing operation.

[0073] Step S106, inputting the fusion features into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion features to obtain a cancer treatment efficacy prediction result.

[0074] The executing entity can input the fusion features into the pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion features and obtain the cancer treatment efficacy prediction results.

[0075] In the specific implementation, the executing entity inputs the fusion features into the cancer treatment efficacy prediction model for high-dimensional mapping to obtain the high-dimensional mapping vector of the fusion features, and then classifies and predicts the high-dimensional mapping vector of the fusion features based on the preset classification decision function to obtain the cancer treatment efficacy prediction result.

[0076] In some embodiments, the cancer treatment efficacy prediction model is a deep neural network model with support vector machine as the main body, which is trained using CT scan sample images, pathological sample images, transcriptomics sample data and clinical sample data. Support vector machine (SVM) is a supervised learning model and related learning algorithm for analyzing data in classification and regression analysis, and the support vector machine contains a support vector machine layer. The cancer treatment efficacy prediction model inputs the fused features into the support vector machine layer for high-dimensional mapping, maps the fused features to the Hilbert space, and obtains a high-dimensional mapping vector of the fused features. Among them, the Hilbert space is an abstract space composed of several independent coordinates, which refers to a complete inner product space. The fused features are mapped to the Hilbert space for inner product calculation to obtain a high-dimensional mapping vector of the fused features.

[0077] Figure 5 is a flowchart of a specific method of step S106 provided in an embodiment of the present application. Figure 5 In some embodiments, the method includes but is not limited to steps S501 to S503.

[0078] Step S501 , using a preset kernel function, in the cancer treatment efficacy prediction model, the fusion feature is mapped to a high-dimensional space for linear transformation to obtain a high-dimensional mapping vector of the fusion feature.

[0079] In some embodiments, a preset kernel function is used to determine the projection relationship between the fused feature and the support vector in the high-dimensional space by calculating the similarity between the fused feature and the preset support vector. The fused feature is mapped to the high-dimensional space in combination with the similarity measure and the position of the support vector in the high-dimensional space to obtain a high-dimensional mapping vector of the fused feature.

[0080] In some embodiments, a kernel function may be preset. Specifically, the preset kernel function may include, but is not limited to, a combination of one or more of the following: polynomial kernel function, radial basis kernel function, Laplace kernel function, Sigmoid kernel function, etc.

[0081] Preferably, a radial basis kernel function is selected as the preset kernel function.

[0082] Step S502, calculating the loss value of each treatment efficacy category based on a preset classification decision function.

[0083] Among them, there are two categories of treatment efficacy, namely PCR category and non-PCR category. The PCR category represents the absence of tumor residue after cancer treatment, and the non-PCR category represents the presence of tumor residue after cancer treatment.

[0084] Step S503, classifying and predicting the high-dimensional mapping vector based on the loss value of the treatment efficacy category to obtain a cancer treatment efficacy prediction result.

[0085] In a specific implementation, when the high-dimensional mapping vector is classified and predicted as the PCR category based on the loss value of the treatment efficacy category, a cancer treatment efficacy prediction result is obtained, which indicates that the current patient's condition has been alleviated. When the high-dimensional mapping vector is classified and predicted as the non-PCR category based on the loss value of the treatment efficacy category, a cancer treatment efficacy prediction result is obtained, which indicates that the current patient's condition has not been alleviated.

[0086] In some embodiments, the above method further comprises: calculating the patient's survival probability based on the cancer treatment efficacy prediction result and performing a significance analysis on the survival probability to obtain a survival analysis result.

[0087] In the specific implementation, after obtaining the prediction results of cancer treatment efficacy, the survival probability of the patient is calculated using the survival probability calculation formula, and the Kaplan-Meier survival curve is drawn. The log-rank test is used to measure whether the difference in survival distribution between the PCR category and the non-PCR category is statistically significant (P-Value<0.05). Finally, based on the shape of the KM curve and the inter-group comparison results, the survival of the pCR category and the non-pCR category is explained to provide clinical decision support.

[0088] The formula for calculating the survival probability is: , in, is the survival probability at the kth time point, is the survival probability at the k-1th time point, is the historical number of deaths at the kth time point, is the sum of the historical number of deaths and the historical number of survivors at the kth time point.

[0089] See also Figure 6 The present application also provides a cancer treatment efficacy prediction device, which can implement the above-mentioned cancer treatment efficacy prediction method, and the device includes: The first module 601 is used to obtain CT scan images, pathological images, transcriptomics data and clinical data; The second module 602 is used to extract radiomics features from the CT scan image to obtain radiomics features; The third module 603 is used to extract pathological features of interest based on the attention mechanism and extract local pathological features based on the results of the pathological features of interest to obtain pathological omics features; The fourth module 604 is used to perform differentially expressed gene analysis on the transcriptomic data and screen differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features; The fifth module 605 is used to fuse radiomic features, pathological features, transcriptomic features and clinical data to obtain fusion features; The sixth module 606 is used to input the fusion features into the pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion features and obtain the cancer treatment efficacy prediction result.

[0090] The specific implementation of the cancer treatment efficacy prediction device is basically the same as the specific implementation of the above-mentioned cancer treatment efficacy prediction method, and will not be repeated here.

[0091] Figure 7 It is a block diagram of an electronic device according to an exemplary embodiment.

[0092] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present disclosure is described. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0093] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), a display unit 740, etc.

[0094] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present disclosure described in the above cancer treatment efficacy prediction method section of this specification.

[0095] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0096] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0097] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0098] The electronic device 700 may also communicate with one or more external devices 700' (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 760. The network adapter 760 may communicate with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0099] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned cancer treatment efficacy prediction method is implemented.

[0100] The cancer treatment efficacy prediction method, device, equipment and storage medium provided in the embodiments of the present application obtain the patient's CT scan image, pathological image, transcriptomic data and clinical data, extract corresponding features to obtain imaging omics features, pathological omics features, transcriptomic features and clinical data, and input the fusion features obtained by fusing imaging omics features, pathological omics features, transcriptomic features and clinical data into the cancer treatment efficacy prediction model, so as to evaluate the cancer treatment efficacy based on the fusion features and obtain the cancer treatment efficacy prediction result. Due to the combination of imaging omics features, pathological omics features, transcriptomic features and clinical features, the cancer treatment efficacy prediction is performed based on multiple modal information, which can improve the accuracy of cancer treatment efficacy prediction.

[0101] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above method according to the implementation of the present disclosure.

[0102] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0103] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0104] Those skilled in the art will appreciate that the above modules can be distributed in the device according to the description of the embodiment, or can be changed accordingly and only used in one or more devices different from the embodiment. The modules of the above embodiments can be combined into one module, or further divided into multiple sub-modules.

[0105] The exemplary embodiments of the present disclosure are specifically shown and described above. It should be understood that the present disclosure is not limited to the detailed structures, configurations or implementations described herein; on the contrary, the present disclosure is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.

Claims

1. A method for predicting the efficacy of cancer treatment, characterized in that: include: Obtain CT scan images, pathological images, transcriptomic data, and clinical data; Extracting radiomics features from the CT scan image to obtain radiomics features; Extracting pathological features of interest based on an attention mechanism and extracting local pathological features based on the results of the pathological features of interest from the pathological image to obtain pathological omics features; Performing differentially expressed gene analysis on the transcriptomic data and screening differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features; fusing the imaging omics features, the pathological omics features, the transcriptomics features, and the clinical data to obtain a fusion feature; The fusion feature is input into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion feature to obtain a cancer treatment efficacy prediction result.

2. The method for predicting the efficacy of cancer treatment according to claim 1, characterized in that: The step of extracting radiomics features from the CT scan image to obtain radiomics features includes: Segmenting the CT scan image into regions of interest to obtain a plurality of CT scan image blocks; Performing radiomics feature extraction on the CT scan image block to obtain the radiomics feature; the radiomics feature extraction includes extracting first-order statistical features, regional shape and size features, and texture features in the CT scan image block.

3. The method for predicting the efficacy of cancer treatment according to claim 1, characterized in that: The pathological image is subjected to the extraction of pathological features of interest based on the attention mechanism and the extraction of local pathological features based on the results of the pathological features of interest extraction to obtain pathological omics features, including: Segmenting the pathological image to obtain a plurality of pathological image blocks; Based on the attention mechanism, the pathological features of interest in the pathological image blocks are sensed to select a number of the pathological image blocks with the highest attention scores to obtain a number of candidate image blocks; the pathological features of interest are pathological features that characterize no tumor residue after cancer treatment; Local pathological features are extracted from the candidate image blocks to obtain the pathological omics features.

4. The method for predicting the therapeutic efficacy of cancer according to claim 3, characterized in that: Based on the attention mechanism, the pathological features of interest in the pathological image blocks are sensed to select several pathological image blocks with the highest attention scores to obtain several candidate image blocks, including: Performing encoding processing on the pathological image block to obtain an encoding output vector; Performing full-connection dimensionality reduction processing on the encoded output vector to obtain a full-connection output vector; According to the importance of the pathological features of interest in the pathological image blocks, an attention weight operation is performed on the fully connected output vector to obtain a number of attention scores, and a number of the pathological image blocks with the highest attention evaluation scores are selected to obtain a number of candidate image blocks.

5. The method for predicting the efficacy of cancer treatment according to claim 1, characterized in that: The step of performing differentially expressed gene analysis on the transcriptomic data and screening differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features includes: Comparing the transcriptomic data with reference genome data to obtain a gene expression matrix corresponding to the transcriptomic data; Performing differential expression analysis on the gene expression matrix for cancer treatment efficacy at the gene level, and screening out a number of differentially expressed genes according to the differential expression analysis results; Linear regression analysis is performed on the gene expression values ​​corresponding to the differentially expressed genes to screen out the differentially expressed genes that meet the linear regression analysis conditions and obtain the transcriptomic features.

6. The method for predicting the efficacy of cancer treatment according to claim 1, characterized in that: The step of inputting the fusion feature into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion feature to obtain a cancer treatment efficacy prediction result includes: Using a preset kernel function, in the cancer treatment efficacy prediction model, the fusion feature is mapped to a high-dimensional space for linear transformation to obtain a high-dimensional mapping vector of the fusion feature; Calculate the loss value for each treatment efficacy category based on a preset classification decision function; The high-dimensional mapping vector is classified and predicted based on the loss value of the treatment efficacy category to obtain the cancer treatment efficacy prediction result.

7. The method for predicting the efficacy of cancer treatment according to claim 1, characterized in that: Also includes: According to the cancer treatment efficacy prediction result, the patient's survival probability is calculated and a significance analysis is performed on the survival probability to obtain a survival analysis result.

8. A cancer treatment efficacy prediction device, characterized in that: include: The first module is used to obtain CT scan images, pathological images, transcriptomic data and clinical data; The second module is used to extract radiomics features from the CT scan image to obtain radiomics features; The third module is used to extract the pathological features of interest based on the attention mechanism and extract the local pathological features based on the results of the pathological features of interest to obtain the pathological omics features; The fourth module is used to perform differentially expressed gene analysis on the transcriptomic data and screen differentially expressed genes based on the differentially expressed gene analysis results to obtain transcriptomic features; A fifth module is used to fuse the radiomics feature, the pathological features, the transcriptomics feature and the clinical data to obtain a fusion feature; The sixth module is used to input the fusion features into a pre-trained cancer treatment efficacy prediction model to evaluate the cancer treatment efficacy based on the fusion features to obtain a cancer treatment efficacy prediction result.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the cancer treatment efficacy prediction method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for predicting the efficacy of cancer treatment according to any one of claims 1 to 7 is implemented.

Images