Tumor multi-gene detection method based on hyperspectral imaging

Abstract

The present invention discloses a method for detecting tumor multi-genes based on hyperspectral images, which belongs to the technical field of mutation sample detection, and specifically includes: obtaining tumor tissue sections of different carcinogenic driver gene mutation states after H&E staining; photographing tumor tissue sections by a hyperspectral camera, obtaining hyperspectral images of tumor samples and preprocessing; building a global feature extractor based on S3Anet; building a tumor multi-gene detection model and using multi-task learning training; collecting tumor samples to be detected, and obtaining hyperspectral data of the samples to be detected by a hyperspectral camera; inputting the hyperspectral data into a trained S3ANet model for feature extraction, and then inputting it into a trained multi-gene detection model to obtain the gene detection results of the tumor samples. The beneficial effects of the present invention are: the present invention effectively reduces the dependence on traditional pathological diagnosis experience and gene sequencing, significantly improves the accuracy, reliability and generalization ability of detection, and reduces the detection cost.

Description

Technical Field

[0001] The present invention relates to the technical field of mutation sample detection, and in particular to a tumor multi-gene detection method based on hyperspectral images. Background Art

[0002] Under the background of existing medical technology, targeted tumor therapy is of great significance for improving patient survival rates. Before conducting targeted tumor therapy, it is usually necessary to detect oncogenic driver genes. At present, the commonly used genetic testing methods mainly include immunohistochemistry (immunohistochemistry technology) and gene sequencing. However, these two methods have many shortcomings. Immunohistochemistry and gene sequencing often take a long time to test, are costly, and consume a lot of human resources, making it difficult to meet the needs of hospitals and patients for rapid and low-cost genetic testing. Therefore, the existing detection methods are limited in wide application, and a new, efficient and economical tumor gene detection method is urgently needed. Summary of the invention

[0003] The purpose of the present invention is to provide a tumor multi-gene detection method based on hyperspectral images to solve the problems of long detection time, high cost, and limited application scope of gene detection methods in the prior art. The present invention uses the hyperspectral image information of tumor slices as training data, builds a model and completes the training, which can accurately identify the mutation level of oncogenic driver genes in patient tissue samples, thereby providing an efficient, low-cost and accurate gene detection method for targeted tumor treatment.

[0004] In order to achieve the above-mentioned object of the invention, the present invention provides a method for detecting multiple genes of tumors based on hyperspectral images, and the detection method comprises the following steps:

[0005] Step S1: Collect tumor tissue samples, obtain tumor tissue sections with different oncogenic driver gene mutation states after H&E staining, and store them at low temperatures;

[0006] Step S2: Acquire the spectral characteristics of ambient light, use a non-fluorescent, white diffuse reflective material as a reference, collect hyperspectral data of the reference area, and obtain an ambient light characteristic curve through averaging;

[0007] Step S3: thawing the tumor tissue sample, photographing the tumor tissue slice with a hyperspectral camera, and obtaining a hyperspectral image of the tumor sample, wherein the hyperspectral image of the tumor sample includes a spectral dimension and a stereoscopic spatial dimension;

[0008] Step S4: preprocessing the hyperspectral image of the tumor sample;

[0009] Step S5: building a global feature extractor based on S3Anet, where S3Anet is a basic structural framework combining a 3D convolutional neural network with a self-attention mechanism, which is used to extract the morphological and optical features of tumor samples, and train the model through a self-supervised learning task to automatically extract features of hyperspectral data;

[0010] Step S6: Build a tumor multi-gene detection model and use multi-task learning training. The tumor multi-gene detection model uses the feature vector extracted by S3ANet to perform binary classification on the mutation status of each gene through multiple task-specific layers to optimize the detection results of multiple genes;

[0011] Step S7: collecting a tumor sample to be detected, and obtaining hyperspectral data of the sample to be detected by a hyperspectral camera;

[0012] Step S8: The hyperspectral data obtained in step S7 is input into the trained S3ANet model for feature extraction, and the obtained feature vector is input into the trained multi-gene detection model, which is used to predict the gene mutation status of the tumor sample to be detected and output the gene detection result of the tumor sample.

[0013] The tumor tissue samples collected in step S1 include tumor sections carrying positive and negative EGFR gene mutations, ALK, FGFR1, PIK3CA, KRAS, ERCC1, RRM1 and HER2 gene mutations, and the samples are derived from normal tissues free from the postoperative margin of surgical patients. The acquisition of the tumor tissue samples does not change the surgical procedure or treatment plan, all sample collections are authorized by the patients, and the gene mutation status of the sample source patients is not determined before surgery.

[0014] The pre-processing of step S4 includes:

[0015] Step S401: using the ambient light characteristic curve obtained in step S2, performing environmental noise reduction processing on the hyperspectral data of the tumor sample, subtracting the ambient light spectrum characteristics, and obtaining the hyperspectral data of the tumor sample after noise reduction;

[0016] Step S402: normalizing the denoised hyperspectral data of the tumor sample, converting the spectral values ​​to the range of [0, 1], and improving data consistency through a min-max normalization method;

[0017] Step S403: performing data enhancement processing, including generating new training samples by image rotation to enhance the generalization ability of the model;

[0018] Step S404: Divide the hyperspectral image into small three-dimensional data blocks, each of which contains a certain number of adjacent pixels and information of all spectral channels, so as to improve memory utilization and later training efficiency, and use the stochastic gradient descent method to accelerate the model training process and avoid overfitting.

[0019] The normalization process in step S402 is performed by calculating the minimum and maximum values ​​in the entire data set, subtracting the minimum value from the spectral value of each pixel and dividing the result by the difference between the maximum value and the minimum value.

[0020] The data enhancement in step S403 processes the original image by randomly rotating the angle to simulate the change of viewing angle in actual applications, so as to improve the robustness of the model to different image forms.

[0021] The S3ANet network includes a three-dimensional convolutional neural network (3D-CNN) module, which consists of multiple convolutional layers, global average pooling layers and fully connected layers;

[0022] Convolutional layer, used to extract the features of spatial and spectral dimensions in hyperspectral images;

[0023] The global average pooling layer maps the extracted features into a feature vector of fixed length;

[0024] The fully connected layer is used to extract and compress global features to facilitate feature expression in classification tasks.

[0025] The S3ANet network also includes a self-attention mechanism module, which generates a feature weight matrix by calculating the similarity between hyperspectral image features, and uses the matrix to weight the hyperspectral image features to highlight important features and suppress redundant information.

[0026] The S3ANet network is trained by self-supervised learning, which includes an image reconstruction task, specifically: a hyperspectral image is input into the S3ANet network, and after being processed by a 3D-CNN module and a self-attention mechanism module, a reconstructed hyperspectral image is output. The reconstruction task optimizes the network parameters by calculating the spectral similarity between the input image and the reconstructed image until the reconstruction error is minimized.

[0027] The multi-task learning model in step S6 is used to simultaneously predict the mutation status of multiple tumor genes, and the multi-task learning model includes an input layer, a shared layer, and a task-specific layer;

[0028] The shared layer is used to extract common features of multiple gene classification tasks;

[0029] The task-specific layer is independently constructed for each gene task, including a fully connected layer and a sigmoid activation function, which is used for binary classification prediction of the mutation status of the gene.

[0030] The multi-task learning model is trained by optimizing the independent loss function of each gene task, and the final loss is the weighted sum of all task losses. The training adopts batch gradient descent method, and the accuracy, F1 score, recall rate and precision of the model are evaluated through the validation set.

[0031] The beneficial effects of the present invention are as follows: the present invention realizes the detection of multiple tumor genes through hyperspectral imaging technology, global feature extraction and self-supervision of the S3ANet model, and multi-task learning methods, effectively reducing the dependence on traditional pathological diagnosis experience and gene sequencing, significantly improving the accuracy, reliability and generalization ability of detection, while reducing the detection cost, and also optimizing the system's computing resource consumption by improving memory utilization and training efficiency, making the detection method suitable for large-scale data processing. Overall, the present invention provides an efficient and economical solution for tumor gene typing and personalized treatment, which helps to improve the early screening rate of cancer and patient prognosis; specifically:

[0032] 1. The present invention uses hyperspectral imaging technology combined with automated processing procedures to quickly obtain spectral and spatial information of tumor slices, and then perform tumor gene typing. Doctors can quickly identify the genetic characteristics of tumors based on the test results and choose the most suitable treatment plan for patients, especially immunosuppressants, thereby shortening the diagnosis time and improving treatment efficiency.

[0033] 2. The use of automated processing methods reduces the reliance on the clinical experience of pathologists. Through the systematic tumor gene testing process, the subjective bias in the clinical diagnosis process is significantly reduced, the accuracy of diagnosis is improved, and the risk of over-treatment or under-treatment of patients is avoided.

[0034] 3. The rapid gene detection method adopted by the present invention can effectively support the early detection and treatment of cancer, thereby reducing the serious health risks caused by cancer to the public.

[0035] 4. The present invention adopts the S3ANet model that combines 3D convolutional neural network (3D-CNN) and self-attention mechanism, which can integrate spectral and spatial information and extract global features of tumor samples. This model not only improves the accuracy of tumor gene detection, but also enhances the reliability of the system.

[0036] 5. By dividing the hyperspectral image into small three-dimensional data blocks and performing data randomization processing, the present invention significantly improves memory utilization efficiency, shortens model training time, and reduces the consumption of computing resources, making it suitable for large-scale data processing.

[0037] 6. The present invention reduces the reliance on a large amount of labeled data through self-supervised learning technology, so that the model can be effectively trained under the condition of less labeled data. At the same time, the application of self-supervised learning improves the adaptability of the model, which can better cope with new data and new environments, and ensure the accuracy and reliability of the detection results.

[0038] 7. The present invention uses multi-task learning to optimize the detection tasks of multiple gene states. By sharing the feature extraction layer, the training efficiency of the model is improved, and the gene states are accurately classified through multiple task-specific layers, further improving the accuracy and consistency of the detection results.

[0039] 8. Through accurate tumor gene typing results, the present invention lays the foundation for personalized medicine, so that treatment plans can be customized according to the patient's specific genetic characteristics, greatly improving the treatment effect and increasing the patient's survival rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a schematic diagram of the tumor multi-gene detection process of embodiments 1 and 2 of the present invention;

[0041] Figure 2 It is a schematic diagram of the data preprocessing process in Embodiments 1 and 2 of the present invention;

[0042] Figure 3 Schematic diagram of the S3ANet neural network architecture used for hyperspectral image feature extraction in embodiments 1 and 2 of the present invention. DETAILED DESCRIPTION

[0043] In order to clearly illustrate the technical features of this solution, this solution is described below through a specific implementation method.

[0044] Embodiment 1

[0045] The embodiment of the present invention provides a method for multi-gene detection of tumors based on hyperspectral images, which comprises the following steps:

[0046] Step S1: Collect tumor tissue samples, obtain tumor tissue sections with different oncogenic driver gene mutation states after H&E staining, and store them at low temperatures;

[0047] Step S2: Acquire the spectral characteristics of ambient light, use a non-fluorescent, white diffuse reflective material as a reference, collect hyperspectral data of the reference area, and obtain an ambient light characteristic curve through averaging;

[0048] Step S3: thawing the tumor tissue sample, photographing the tumor tissue slice with a hyperspectral camera, and obtaining a hyperspectral image of the tumor sample, wherein the hyperspectral image of the tumor sample includes a spectral dimension and a stereoscopic spatial dimension;

[0049] Step S4: preprocessing the hyperspectral image of the tumor sample;

[0050] Step S5: building a global feature extractor based on S3Anet, where S3Anet is a basic structural framework combining a 3D convolutional neural network with a self-attention mechanism, which is used to extract the morphological and optical features of tumor samples, and train the model through a self-supervised learning task to automatically extract features of hyperspectral data;

[0051] Step S6: Build a tumor multi-gene detection model and use multi-task learning training. The tumor multi-gene detection model uses the feature vector extracted by S3ANet to perform binary classification on the mutation status of each gene through multiple task-specific layers to optimize the detection results of multiple genes;

[0052] Step S7: collecting a tumor sample to be detected, and obtaining hyperspectral data of the sample to be detected by a hyperspectral camera;

[0053] Step S8: The hyperspectral data obtained in step S7 is input into the trained S3ANet model for feature extraction, and the obtained feature vector is input into the trained multi-gene detection model, which is used to predict the gene mutation status of the tumor sample to be detected and output the gene detection result of the tumor sample.

[0054] The tumor tissue samples collected in step S1 include tumor sections carrying positive and negative EGFR gene mutations, ALK, FGFR1, PIK3CA, KRAS, ERCC1, RRM1 and HER2 gene mutations, and the samples are derived from normal tissues free from the postoperative margin of surgical patients. The acquisition of the tumor tissue samples does not change the surgical procedure or treatment plan, all sample collections are authorized by the patients, and the gene mutation status of the sample source patients is not determined before surgery.

[0055] The pre-processing of step S4 includes:

[0056] Step S401: using the ambient light characteristic curve obtained in step S2, performing environmental noise reduction processing on the hyperspectral data of the tumor sample, subtracting the ambient light spectrum characteristics, and obtaining the hyperspectral data of the tumor sample after noise reduction;

[0057] Step S402: normalizing the denoised hyperspectral data of the tumor sample, converting the spectral values ​​to the range of [0, 1], and improving data consistency through a min-max normalization method;

[0058] Step S403: performing data enhancement processing, including generating new training samples by image rotation to enhance the generalization ability of the model;

[0059] Step S404: Divide the hyperspectral image into small three-dimensional data blocks, each of which contains a certain number of adjacent pixels and information of all spectral channels, so as to improve memory utilization and later training efficiency, and use the stochastic gradient descent method to accelerate the model training process and avoid overfitting.

[0060] The normalization process in step S402 is performed by calculating the minimum and maximum values ​​in the entire data set, subtracting the minimum value from the spectral value of each pixel and dividing the result by the difference between the maximum value and the minimum value.

[0061] The data enhancement in step S403 processes the original image by randomly rotating the angle to simulate the change of viewing angle in actual applications, so as to improve the robustness of the model to different image forms.

[0062] The S3ANet network includes a three-dimensional convolutional neural network (3D-CNN) module, which consists of multiple convolutional layers, global average pooling layers and fully connected layers;

[0063] Convolutional layer, used to extract the features of spatial and spectral dimensions in hyperspectral images;

[0064] The global average pooling layer maps the extracted features into a feature vector of fixed length;

[0065] The fully connected layer is used to extract and compress global features to facilitate feature expression in classification tasks.

[0066] The S3ANet network also includes a self-attention mechanism module, which generates a feature weight matrix by calculating the similarity between hyperspectral image features, and uses the matrix to weight the hyperspectral image features to highlight important features and suppress redundant information.

[0067] The S3ANet network is trained by self-supervised learning, which includes an image reconstruction task, specifically: a hyperspectral image is input into the S3ANet network, and after being processed by a 3D-CNN module and a self-attention mechanism module, a reconstructed hyperspectral image is output. The reconstruction task optimizes the network parameters by calculating the spectral similarity between the input image and the reconstructed image until the reconstruction error is minimized.

[0068] The multi-task learning model in step S6 is used to simultaneously predict the mutation status of multiple tumor genes, and the multi-task learning model includes an input layer, a shared layer, and a task-specific layer;

[0069] The shared layer is used to extract common features of multiple gene classification tasks;

[0070] The task-specific layer is independently constructed for each gene task, including a fully connected layer and a sigmoid activation function, which is used for binary classification prediction of the mutation status of the gene.

[0071] The multi-task learning model is trained by optimizing the independent loss function of each gene task, and the final loss is the weighted sum of all task losses. The training adopts batch gradient descent method, and the accuracy, F1 score, recall rate and precision of the model are evaluated through the validation set.

[0072] Embodiment 2

[0073] like Figure 1-Figure 2 As shown, the embodiment of the present invention provides a method for multi-gene detection of tumors based on hyperspectral imaging, which specifically includes the following steps:

[0074] Step S1: Collect tumor tissue samples from patients undergoing tumor surgery and store the samples at low temperatures.

[0075] In this embodiment, hyperspectral image information of tumor tissue samples is obtained as training data. The tumor slices used include tumor EGFR gene mutation-positive slices and tumor EGFR gene mutation-negative slices and other tumor slices with different mutations of carcinogenic driver genes. The use of all samples is authorized, and the collection of samples does not require changes in surgical procedures or treatment plans. The source of the samples is not determined before surgery. The samples are all derived from normal tissues free at the edge of the tumor tissue after surgery. The above slices are all stained with H&E. The pathological results are classified according to the results of immunohistochemistry to form tumor samples with different grades of EGFR, ALK, FGFR1, PIK3CA, KRAS, ERCC1, RRM1, and HER2 gene mutations, and all of them are placed at low temperature.

[0076] Step S2: Obtaining the spectral characteristics of ambient light.

[0077] In this embodiment, during the hyperspectral imaging process, a piece of non-fluorescent, white diffuse reflective material is used as a reference and placed on the imaging plane. Hyperspectral imaging is performed on the reference area to collect one-dimensional spectral data of the area; and then the collected spectral data are averaged to obtain a spectral characteristic curve representing the ambient light conditions.

[0078] Step S3: Scanning early tumor tissue samples based on the hyperspectral spectrum.

[0079] After the tissue sample processed in step S1 is thawed, it is photographed by a hyperspectral camera to obtain a hyperspectral original image. In this embodiment, the hyperspectral image includes a spectral dimension and a stereoscopic spatial dimension, and the hyperspectral data is collected at the same time.

[0080] Step S4: preprocess the hyperspectral image.

[0081] Firstly, the spectral characteristics of ambient light are subtracted from the hyperspectral image to achieve environmental noise reduction, that is, the one-dimensional spectral data of each pixel in the tumor hyperspectral image is subtracted from the average spectral characteristics of ambient light.

[0082] To ensure that the data is in the same range, normalization is performed. This process involves converting the spectral value of each pixel to the range of [0, 1]. The specific operation is to first calculate the minimum and maximum values ​​in the entire data set, then subtract the minimum value from the spectral value of each pixel, and finally divide the result by the difference between the maximum and minimum values. This min-max normalization method helps to unify the scale of different data sources, allowing the model to treat each pixel more fairly during training, thereby improving data consistency.

[0083] In order to enhance the generalization ability of the model, this embodiment also adopts data enhancement technology, that is, generating new training samples by rotating the original image, which can simulate the perspective changes that may occur in the real world and increase the diversity of data, thereby helping the model learn more robust features.

[0084] When training deep learning models, hyperspectral images are divided into small three-dimensional data blocks. This not only improves memory utilization, but also accelerates the model training process through stochastic gradient descent in each batch.

[0085] In order to prevent the model from becoming dependent on the specific order of data during training, which can lead to overfitting, the data needs to be randomly shuffled after each training cycle (epoch). This method of data randomization ensures that the order of samples is different each time during training, forcing the model to learn more generalized features instead of just remembering a specific data order.

[0086] Step S5: Extract image global features based on S3ANet.

[0087] In order to extract as many image morphological and optical features as possible, this embodiment innovatively proposes S3ANet, which combines the 3D-CNN network that can effectively process hyperspectral images and the self-attention mechanism that focuses on morphological optical features to form a new network architecture. The feature extractor that can automatically extract features on unlabeled hyperspectral data is obtained through the self-supervised task training model; specifically:

[0088] (1) Model construction:

[0089] Use 3D convolutional neural network to extract spatial and spectral features in hyperspectral images. 3D-CNN can process the spatial and spectral information of images at the same time and capture more fine-grained features. On the basis of 3D-CNN, add self-attention mechanism to enhance the focus on important image morphology and optical features. The self-attention mechanism calculates the similarity between features and selectively focuses on more important features, thereby improving the accuracy of feature extraction.

[0090] (2) Network structure:

[0091] Multiple layers of 3D convolutional layers are used to extract low-level and high-level features of the image.

[0092] The global average pooling layer is used to map features to a vector of fixed length for subsequent processing.

[0093] Self-attention layer to enhance the interaction and importance between features.

[0094] The fully connected layer is used to further extract global features.

[0095] (3) Self-supervised learning task training model:

[0096] A self-supervised learning task of image reconstruction is set to train the model. The purpose of the self-supervised learning task is to train the model with unlabeled data so that it can learn effective feature representation. In the image reconstruction task, the goal of the model is to reconstruct the input hyperspectral image. By minimizing the difference between the reconstructed image and the original image, the model can learn the effective features of the input data.

[0097] Training process:

[0098] Splitting the dataset into training and validation sets ensures that the model can be validated and tuned during the training process.

[0099] Use the self-supervised learning task described above to train the model. Use the back-propagation algorithm to continuously adjust the model's parameters so that it can accurately reconstruct the input image. After each training iteration, evaluate the model's performance on the validation set to ensure that the model can effectively learn the characteristics of the data.

[0100] (4) Feature extraction:

[0101] After training, a feature extractor is used to extract global features from the hyperspectral image. These feature vectors can be used for subsequent multi-task learning.

[0102] Step S6: Use multi-task learning to train a tumor multi-gene detection model.

[0103] (1) Data preparation:

[0104] Obtain labeled data, including the negative or positive status of the six genes for each sample. The status of each gene is represented by 0 (negative) and 1 (positive). Use the trained S3ANet model to extract features from the hyperspectral image, which will be used as the input of the multi-task learning model.

[0105] (2) Building a multi-task learning model:

[0106] The network architecture is:

[0107] Input layer: accepts feature vectors extracted from S3ANet;

[0108] Shared layer: one or more shared fully connected layers used to extract common features;

[0109] Task-specific layers: A set of task-specific layers is constructed for each gene separately, using fully connected layers and sigmoid activation functions for binary classification.

[0110] (3) Data division:

[0111] The dataset is divided into a training set and a validation set. The training set is used for model training, and the validation set is used for model evaluation and tuning.

[0112] (4) Training model:

[0113] Use a multi-task learning approach to optimize the classification tasks of all genes simultaneously. The output layer of each gene is optimized using an independent loss function, and the total loss is the weighted sum of all task losses. Use batch gradient descent to train the model and minimize the total loss. After each training iteration, evaluate the performance of the model on the validation set and perform necessary hyperparameter tuning.

[0114] (5) Model evaluation:

[0115] The model performance was evaluated using metrics such as accuracy, F1 score, recall, and precision.

[0116] The classification results for each gene were evaluated individually and the overall model performance was calculated.

[0117] The model is evaluated on the validation set to ensure the generalization ability and robustness of the model.

[0118] Embodiment 3

[0119] The embodiment of the present invention provides a system for implementing the above-mentioned tumor multi-gene detection method based on hyperspectral image in embodiment 1 or 2, and the system includes:

[0120] A sample collection module is used to collect tumor tissue samples, obtain tumor tissue sections with different mutation states of oncogenic driver genes after H&E staining, and store them at low temperatures;

[0121] The spectrum acquisition module is used to obtain the spectral characteristics of the ambient light. It uses a non-fluorescent, white diffuse reflective material as a reference object to collect the hyperspectral data of the reference area, and obtains the ambient light characteristic curve through averaging processing;

[0122] A hyperspectral imaging module is used to thaw tumor tissue samples and photograph tumor tissue sections using a hyperspectral camera to obtain hyperspectral images of tumor samples that include spectral and stereoscopic spatial dimensions;

[0123] An image preprocessing module, used for preprocessing the hyperspectral images of tumor samples;

[0124] The global feature extraction module is built on S3ANet, which is a basic structural framework that combines 3D convolutional neural networks with self-attention mechanisms. It is used to extract the morphological and optical features of tumor samples and train the model through self-supervised learning tasks to automatically extract features of hyperspectral data.

[0125] The tumor multi-gene detection module is used to train the feature vectors extracted by S3ANet using multi-task learning, and to classify the mutation status of each gene through multiple task-specific layers to optimize the detection results of multiple genes;

[0126] A sample detection module is used to collect hyperspectral data of a tumor sample to be detected and obtain a hyperspectral image of the sample to be detected through a hyperspectral camera;

[0127] The data input and prediction module is used to input the hyperspectral data of the sample to be tested into the trained S3ANet model for feature extraction, and input the extracted feature vector into the trained tumor multi-gene detection model to predict the gene mutation status of the tumor sample to be tested and output the gene detection results of the tumor sample.

[0128] Embodiment 4

[0129] This embodiment provides a computer-readable storage medium having a computer program stored thereon, which can implement the steps in the prostate tumor identification method of embodiments one and two when executed by a processor, such as the contents described in the above embodiments one and two.

[0130] Embodiment 5

[0131] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, which implements the steps in the prostate tumor identification method based on hyperspectral images described in the above-mentioned embodiments 1 and 2, and is implemented when the processor executes the program.

[0132] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for multi-gene detection of tumors based on hyperspectral images, characterized in that: The detection method comprises the following steps: Step S1: Collect tumor tissue samples, obtain tumor tissue sections with different oncogenic driver gene mutation states after H&E staining, and store them at low temperatures; Step S2: Acquire the spectral characteristics of ambient light, use a non-fluorescent, white diffuse reflective material as a reference, collect hyperspectral data of the reference area, and obtain an ambient light characteristic curve through averaging; Step S3: thawing the tumor tissue sample, photographing the tumor tissue slice with a hyperspectral camera, and obtaining a hyperspectral image of the tumor sample, wherein the hyperspectral image of the tumor sample includes a spectral dimension and a stereoscopic spatial dimension; Step S4: preprocessing the hyperspectral image of the tumor sample; Step S5: building a global feature extractor based on S3Anet, where S3Anet is a basic structural framework combining a 3D convolutional neural network with a self-attention mechanism, which is used to extract the morphological and optical features of tumor samples, and train the model through a self-supervised learning task to automatically extract features of hyperspectral data; Step S6: Build a tumor multi-gene detection model and use multi-task learning training. The tumor multi-gene detection model uses the feature vector extracted by S3ANet to perform binary classification on the mutation status of each gene through multiple task-specific layers to optimize the detection results of multiple genes; Step S7: collecting a tumor sample to be detected, and obtaining hyperspectral data of the sample to be detected by a hyperspectral camera; Step S8: The hyperspectral data obtained in step S7 is input into the trained S3ANet model for feature extraction, and the obtained feature vector is input into the trained multi-gene detection model, which is used to predict the gene mutation status of the tumor sample to be detected and output the gene detection result of the tumor sample.

2. The method for multi-gene detection of tumors based on hyperspectral images according to claim 1, characterized in that: The tumor tissue samples collected in step S1 include tumor sections carrying positive and negative EGFR gene mutations, ALK, FGFR1, PIK3CA, KRAS, ERCC1, RRM1 and HER2 gene mutation status, and the samples are derived from normal tissues free at the postoperative margin of surgical patients.

3. The method for multi-gene detection of tumors based on hyperspectral images according to claim 1, characterized in that: The pre-processing of step S4 includes: Step S401: using the ambient light characteristic curve obtained in step S2, performing environmental noise reduction processing on the hyperspectral data of the tumor sample, subtracting the ambient light spectrum characteristics, and obtaining the hyperspectral data of the tumor sample after noise reduction; Step S402: normalizing the denoised hyperspectral data of the tumor sample, converting the spectral values ​​to the range of [0, 1], and improving data consistency through the min-max normalization method; Step S403: performing data enhancement processing, including generating new training samples by image rotation; Step S404: Divide the hyperspectral image into small three-dimensional data blocks, each of which contains a certain number of adjacent pixels and information of all spectral channels, and use the stochastic gradient descent method to accelerate the model training process.

4. The method for multi-gene detection of tumors based on hyperspectral images according to claim 3, characterized in that: The normalization process in step S402 is performed by calculating the minimum and maximum values ​​in the entire data set, subtracting the minimum value from the spectral value of each pixel and dividing the result by the difference between the maximum value and the minimum value.

5. The method for multi-gene detection of tumors based on hyperspectral images according to claim 3, characterized in that: The data enhancement in step S403 processes the original image by randomly rotating the angle to simulate the change of viewing angle in actual application.

6. The method for multi-gene detection of tumors based on hyperspectral images according to claim 1, characterized in that: The S3ANet includes a three-dimensional convolutional neural network module, which consists of multiple convolutional layers, a global average pooling layer and a fully connected layer; Convolutional layer, used to extract the features of spatial and spectral dimensions in hyperspectral images; The global average pooling layer maps the extracted features into a feature vector of fixed length; The fully connected layer is used to extract and compress global features to facilitate feature expression in classification tasks.

7. The method for multi-gene detection of tumors based on hyperspectral images according to claim 6, characterized in that: The S3ANet also includes a self-attention mechanism module, which generates a feature weight matrix by calculating the similarity between hyperspectral image features and uses the matrix to perform weighted processing on the hyperspectral image features.

8. The method for multi-gene detection of tumors based on hyperspectral images according to claim 7, characterized in that: The S3ANet is trained by self-supervised learning, which includes an image reconstruction task, specifically: a hyperspectral image is input into S3ANet, and after being processed by a three-dimensional convolutional neural network module and a self-attention mechanism module, a reconstructed hyperspectral image is output. The reconstruction task optimizes the network parameters by calculating the spectral similarity between the input image and the reconstructed image until the reconstruction error is minimized.

9. The method for multi-gene detection of tumors based on hyperspectral images according to claim 1, characterized in that: The multi-task learning model in step S6 is used to simultaneously predict the mutation status of multiple tumor genes, and the multi-task learning model includes an input layer, a shared layer, and a task-specific layer; The shared layer is used to extract common features of multiple gene classification tasks; The task-specific layer is independently constructed for each gene task, including a fully connected layer and a sigmoid activation function, which is used for binary classification prediction of the mutation status of the gene.

10. The method for multi-gene detection of tumors based on hyperspectral images according to claim 9, characterized in that: The multi-task learning model is trained by optimizing the independent loss function of each gene task, and the final loss is the weighted sum of all task losses. The training adopts batch gradient descent method, and the accuracy, F1 score, recall rate and precision of the model are evaluated through the validation set.

Images