Oral cancer multi-task diagnosis system based on Raman spectrum

Abstract

The invention discloses an oral cancer multi-task diagnosis system based on a Raman spectrum, and belongs to the technical field of biomedical detection. Comprising an acquisition module and an oral cancer multi-task diagnosis module, and the acquisition module is used for acquiring Raman spectrum data; the oral cancer multi-task diagnosis module is used for processing the Raman spectrum data through a trained oral cancer multi-task diagnosis model so as to generate and output a tumor staging result, a lymph node staging result and a histological staging result; the oral cancer multi-task diagnosis model comprises a bidirectional Mama branch and a residual attention fusion branch, local feature information in the Raman spectrum data is extracted by using the residual attention fusion branch, and multi-scale features are generated by fusing global feature information extracted by the bidirectional Mama branch. Feature information in Raman spectrum data can be effectively extracted, the accuracy of oral cancer diagnosis is improved, and multi-task diagnosis of oral cancer is realized; the problems that multi-task common diagnosis cannot be realized and the diagnosis accuracy needs to be improved in the prior art are solved.

Description

Technical Field

[0001] The invention relates to the technical field of biomedical detection, and in particular to a multi-task diagnosis system for oral cancer based on Raman spectroscopy. Background Art

[0002] The statements in this section merely mention background art related to the present invention and do not necessarily constitute prior art.

[0003] Raman spectroscopy is a fast and easy optical technique that is one of the most widely used techniques for the non-destructive characterization of molecules and materials. Raman spectroscopy can detect the vibrational modes of molecules, is sensitive to their chemical bonds, and provides a unique "fingerprint" that enables the identification of chemical substances, so it can be used for early diagnosis of various types of cancer.

[0004] The classification methods of Raman spectroscopy data are mainly divided into two parts: traditional machine learning and deep learning. Traditional machine learning methods mainly include principal component analysis, linear discriminant analysis, support vector machine, etc. Deep learning methods mainly use various classic neural network models combined with Raman spectroscopy technology to achieve tasks such as tumor staging. Convolutional neural networks extract local structure and position information through stacked convolution operations. Their pooling layers will lose a lot of valuable information, ignore the correlation between the local and the whole, and cannot obtain additional global information. The Transformer model uses a self-attention mechanism and a fully connected network to obtain long-distance dependencies and position information in the data, but its complexity is too large, the training and inference speeds are slow, and the stability of the model is limited.

[0005] In addition, the TNM staging system (tumor stage, lymph node stage, and metastasis) of tumors provides clinicians with a reliable basis for patient prognosis and management decisions, and is the most reliable predictor of survival for oral cancer patients. However, in clinical practice, it is not enough to rely solely on the TNM classification, and it is also necessary to analyze the predictive ability of other prognostic factors, among which histological grade may be an important prognostic factor for treatment outcome. Therefore, in the diagnosis of oral cancer, it is necessary to pay attention not only to tumor staging and lymph node staging, but also to histological grade. All three play a key role in promoting timely treatment and evaluating prognosis of oral cancer.

[0006] However, the focus of existing research is mostly limited to a single task, without exploring the shared feature information of Raman spectroscopy data in the three tasks of tumor staging, lymph node staging and histological grading, which limits the ability to analyze the potential information in the data. Summary of the invention

[0007] In order to address the deficiencies of the prior art, the present invention provides a multi-task oral cancer diagnosis system, electronic device, computer-readable storage medium and computer program product based on Raman spectroscopy, which extracts global feature information through a bidirectional Mamba module, extracts local feature information through a residual attention fusion module and reduces the difference between global features and local features, thereby achieving effective feature extraction of Raman spectral data and improving the accuracy of diagnosis.

[0008] In a first aspect, the present invention provides a multi-task diagnosis system for oral cancer based on Raman spectroscopy; A multi-task diagnosis system for oral cancer based on Raman spectroscopy, comprising: An acquisition module, used for acquiring Raman spectrum data; An oral cancer multi-task diagnosis module, used for processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

[0009] In some embodiments, the bidirectional Mamba branch includes a position encoding layer and a plurality of bidirectional Mamba modules connected in sequence; The position encoding layer is used to add position encoding to the preprocessed Raman spectrum data, and the bidirectional Mamba module is used to perform bidirectional sequence global modeling on the input data.

[0010] In some embodiments, performing bidirectional sequence global modeling on input data specifically includes: Perform layer normalization on the input data and linearly project them respectively to obtain the first sequence and the second sequence with the same dimension; Perform bidirectional SSM processing on the first sequence to obtain a forward sequence and a backward sequence, and fuse them with the second sequence processed by the activation function to obtain updated forward sequence and backward sequence; The updated forward sequence and backward sequence are added and processed by linear projection, and then connected with the input data residual to obtain the output sequence.

[0011] In some embodiments, the residual attention fusion branch includes several residual attention fusion modules, and the residual attention fusion module is used to perform feature fusion on the same-layer output of the bidirectional Mamba branch and the output of the residual attention fusion module of the previous layer.

[0012] In some implementations, the feature fusion of the same-layer output of the bidirectional Mamba branch and the output of the residual attention fusion module of the previous layer specifically includes: Perform layer normalization, convolution, and normalization operations on the same-layer outputs of the bidirectional Mamba branch in sequence to obtain a first output sequence; The output of the residual attention fusion module of the previous layer is layer-normalized and fused with the multi-head attention mechanism to obtain the second output sequence; The first output sequence and the second output sequence are fused, and convolution and normalization operations are performed in sequence, and residual connection is performed with the output of the first-level residual attention fusion module to obtain the fused output sequence.

[0013] In some embodiments, the oral cancer multi-task diagnosis model further includes a multi-task classification head, which is a plurality of parallel multi-layer perceptrons.

[0014] In some embodiments, the oral cancer multi-task diagnosis model is trained using a cross entropy loss function.

[0015] In a second aspect, the present invention provides an electronic device; An electronic device comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the following steps: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

[0016] In a third aspect, the present invention provides a computer-readable storage medium; A computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the following steps: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

[0017] In a fourth aspect, the present invention provides a computer program product; A computer program product, comprising a computer program / instruction, which when executed by a processor implements the following steps: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

[0018] Compared with the prior art, the present invention has the following beneficial effects: 1. The technical solution provided by the present invention processes Raman spectral data with spatial characteristics through a bidirectional Mamba module, extracts global feature information, realizes remote relationship modeling capabilities while ensuring low complexity, avoids the loss of valuable information, and improves the reasoning speed and stability of multi-task diagnosis of oral cancer.

[0019] 2. The technical solution provided by the present invention takes into account the correlation between local information and overall information. The residual attention fusion module can effectively extract feature information in a small area from the Raman spectral data, helping to capture the details of the microstructure in the sample, such as specific peaks and troughs, and thus better capture the subtle changes in the tiny areas of the sample, which helps to distinguish the differences between samples. At the same time, it helps to deal with noise and interference in the data and enhance the robustness of the model.

[0020] 3. The technical solution provided by the present invention introduces a multi-head attention mechanism module to increase the dependency of remote modeling, and generates multi-scale features by fusing the global feature information of the bidirectional Mamba branch, and finally projects them into the same feature space to complete feature fusion and realize the comprehensive extraction of Raman spectroscopy data features. Global features can provide more extensive data background information, reflect the characteristics of the overall sample, help capture the commonality and generality between samples, reduce overfitting, and improve the generalization ability of the model. Local features and global features complement each other and can promote the effective implementation of the modeling process.

[0021] 3. The technical solution provided by the present invention designs a new multi-task network model, which realizes the simultaneous discrimination of three tasks by sharing weights and computational complexity, thereby improving the performance of a single task while reducing the overall computational cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0023] Figure 1 A schematic diagram of the network architecture of a multi-task oral cancer diagnosis system based on Raman spectroscopy provided by an embodiment of the present invention; Figure 2 A schematic diagram of the network architecture of a bidirectional Mamba module provided in an embodiment of the present invention; Figure 3 A schematic diagram of the network architecture of the residual attention fusion module provided in an embodiment of the present invention; Figure 4 The present invention provides a schematic diagram of the overall implementation process of a multi-task oral cancer diagnosis system based on Raman spectroscopy according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0025] It should be noted that the terms used herein are only for describing specific embodiments, and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "have" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0027] Embodiment 1 Due to the location characteristics of Raman spectroscopy data and the influence of contextual data, it does not have time series characteristics. The existing oral cancer diagnosis model is not directly applicable to the modeling of Raman spectroscopy data, which will cause the loss of valuable information and ignore the shared feature information in multi-task diagnosis, affecting the accuracy and efficiency of diagnosis. Therefore, the present invention provides an oral cancer multi-task diagnosis system based on Raman spectroscopy.

[0028] Next, combine Figure 1-Figure 4, a multi-task oral cancer diagnosis system based on Raman spectroscopy disclosed in this embodiment is described in detail, and global feature information is provided by a bidirectional Mamba module to achieve remote relationship modeling capabilities while ensuring low complexity. The residual attention fusion module then uses a multi-head attention mechanism to reduce the difference between global features and local features and fuse them into the same feature space.

[0029] The oral cancer multi-task diagnosis system based on Raman spectroscopy includes an acquisition module and an oral cancer multi-task diagnosis module. The acquisition module is used to acquire Raman spectral data, and the oral cancer multi-task diagnosis module is used to process the Raman spectral data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results and histological staging results.

[0030] Combination Figure 1 The oral cancer multi-task diagnosis model includes an embedding layer, a parallel bidirectional Mamba branch and a residual attention fusion branch, and a multi-task classification head. The bidirectional Mamba branch includes a position encoding layer and several bidirectional Mamba modules connected in sequence. The residual attention fusion branch includes several residual attention fusion modules. The output of the embedding layer is respectively input into the position encoding layer and the first residual attention fusion module, and the input of the remaining residual attention fusion modules is the bidirectional Mamba module of the same layer and the residual attention fusion module of the previous layer.

[0031] Specifically, the preprocessed Raman spectral data is first Input embedding layer, Raman spectroscopy data is transformed by linear projection Perform dimensionality reduction to obtain , then we will Input the bidirectional Mamba branch and the residual attention fusion branch. Add position encoding to the bidirectional Mamba branch to get Then the whole sequence is processed by several layers of bidirectional Mamba modules to get the output ; In the residual attention fusion branch, the entire sequence passes through several layers of residual attention fusion modules. The input of each layer includes the output of the residual attention fusion module of the previous layer and the output of the current bidirectional Mamba module. Finally, the output of this branch is obtained Finally, the output sequences of the two branches are and Fusion is performed and the output sequence is normalized to obtain And sent to the multi-task head, different tasks will have their own multi-layer perceptron head and get the final prediction result At a macro level, the overall process is represented as follows:

[0032]

[0033]

[0034]

[0035]

[0036]

[0037] in, represents a linear projection, represents the position code, Represents a bidirectional Mamba block, Indicates the number of layers of the bidirectional Mamba block, represents the residual attention fusion module, represents the addition operation, Representation layer normalization, Indicates the task The multi-layer perceptron head.

[0038] As an implementation method, the detailed implementation process of processing Raman spectroscopy data through the trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results and histological staging results is as follows: (1) The Raman spectral data sequence is input into the embedding layer to perform the embedding operation, then input into the position encoding layer to add the position encoding, and then input into the four bidirectional Mamba modules for processing in sequence to obtain the final output .

[0039] Next, combine Figure 2 , taking any bidirectional Mamba module as an example, it is explained in detail.

[0040] First, The input sequence of the layer After layer normalization, we get , where the input of layer 0 is , the process is as follows:

[0041]

[0042] In the formula, Representation layer normalization, Represents a linear projection.

[0043] Next, the normalized sequences are linearly projected onto the same dimension size and On, then Bidirectional SSM processing is required, where and same, Depend on Flip it in the feature dimension, the process is as follows:

[0044]

[0045]

[0046]

[0047] In the formula, Represents a flip operation.

[0048] Then we need to process it from both the forward and backward directions. and The processing process in both directions is the same, and the forward direction is used as an example here.

[0049] First, Perform one-dimensional convolution and process it with SiLU activation function to obtain ,Will Send to SSM module, Linear projection gives , and The detailed process is as follows:

[0050]

[0051]

[0052]

[0053] in, represents one-dimensional convolution, Indicates that the SiLU activation function is used for processing. , and Both represent linear projection layers, Indicates time scale Parameters.

[0054] Next, use After discretization, we get and , and finally calculated by the SSM module .

[0055] Based on the same process, we can also get , the process is as follows:

[0056]

[0057]

[0058] in, Represents the state matrix Parameters, represents matrix multiplication, Represents state-space model processing.

[0059] Here, you need to Flip it back again, and then and after flipping Respectively processed by SiLU activation function Perform matrix multiplication to fuse and get and ,Will and Add and linearly project, and then Perform residual connection to get the output sequence The process is as follows:

[0060]

[0061]

[0062]

[0063] In the formula, Indicates that the SiLU activation function is used for processing. represents matrix multiplication, Indicates a flip operation. represents a linear projection, Represents matrix addition.

[0064] (2) At the same time, the output of each bidirectional Mamba module in the bidirectional Mamba branch is also input into the residual attention fusion module of the same layer, and combined with the output of the residual attention fusion module of the previous layer for feature fusion. Figure 3 , the implementation process of the residual attention fusion module is as follows: First, the Output sequence of bidirectional Mamba modules Input to this branch for layer normalization, and then perform convolution and normalization operations to obtain At the same time, Output sequence of layer residual attention fusion module Perform layer normalization and then obtain it through the multi-head attention mechanism fusion module The above process is expressed as follows:

[0065]

[0066] in, Representation layer normalization, represents one-dimensional convolution, represents normalization, Represents the fusion of multi-head attention mechanisms.

[0067] Next, the two branches and Fusion , and perform two more convolution and normalization operations to obtain , the process is as follows:

[0068]

[0069] in, represents one-dimensional convolution, represents normalization, Represents matrix addition.

[0070] Finally, the Output sequence of layer residual attention fusion module and Perform residual connection and get the first Output of the layer , the process is as follows:

[0071] in, Representation layer normalization, Represents matrix addition.

[0072] Considering that the residual attention fusion branch effectively extracts local feature information from Raman spectral data through several residual blocks and convolution operations, a multi-head attention mechanism module is introduced to increase the dependency of long-range modeling, and multi-scale features are generated by fusing the global feature information of the bidirectional Mamba branch. Finally, they are projected into the same feature space to complete feature fusion. This mechanism also ensures the full extraction of local and global features while preventing the model from being too complex.

[0073] Local features and global features are complementary in capturing sample information, which can enable the model to understand and characterize the data more comprehensively and extract comprehensive information related to the multi-task diagnosis of oral cancer. Extracting local features captures subtle changes and features in the local area by performing a large number of convolution operations, allowing the model to analyze the data more carefully. Global features provide an overall understanding of the overall sample characteristics, which helps the model better understand the overall structure and relationship of the data, which can simplify the modeling problem to a certain extent. By combining local features and global features, the model is able to control the complexity of the model while retaining the details and overall structure of the data, thereby improving the accuracy and robustness of Raman spectroscopy modeling and providing accurate and comprehensive basic data for subsequent multi-task diagnosis of oral cancer.

[0074] (3) The final output of the bidirectional Mamba branch and the final output of the residual attention fusion branch are fused and layer normalized, and then input in parallel into three classification heads constructed by multi-layer perceptrons to obtain the final prediction result.

[0075] As an implementation method, before the oral cancer multi-task diagnosis model is applied, the oral cancer multi-task diagnosis model needs to be trained. The specific process is as follows: Step 1: Obtain Raman spectral data of oral cancer patients and construct a training set.

[0076] In this embodiment, with the patient's authorization, the Raman spectral data of oral cancer patients are derived from the Department of Oral and Maxillofacial Surgery of Peking Union Medical College Hospital, Chinese Academy of Sciences; the data are collected by a portable fiber-optic Raman spectrometer prototype, which includes a display, a manual displacement platform, an edge computing device, a semiconductor laser, a spectrometer, and a fiber-optic Raman spectral probe. This embodiment does not make any improvements to it, so it will not be described in detail here.

[0077] During the Raman spectrum acquisition process, the fiber Raman spectrum probe maintains a constant distance of 0.5mm from the surface of the tissue to be tested to ensure the stability of the spectral signal. Before acquisition, the background light is removed when the laser is disabled. The laser is then reactivated to collect data on the tumor and normal tissue surfaces, respectively. For each patient, 3 points are randomly selected on the surface of the tumor tissue and 2 points are randomly selected on the surface of the normal tissue. Spectral data are collected 10 times at each point, with an integration time of 1 second each time. The laser power at the tip of the probe is precisely calibrated to 100mW. The entire detection process is ensured to be completed within 30 minutes after tissue resection. After the Raman spectrum acquisition is completed, the collected tumor tissue and normal tissue samples are subjected to pathological testing.

[0078] Step 2: Perform data preprocessing on the Raman spectroscopy data.

[0079] The wavenumber range of the Raman spectrum collected in this embodiment is 0-4075cm-1 , due to 0-400cm -1 The spectral intensity of the data is too high. To ensure the validity of the data, select 400-4000cm -1 The model is trained based on Raman spectral data within a certain wavenumber range, which helps to capture more useful features of each sample.

[0080] Next, the z-score standardization method is used to process the data to make the data distribution more uniform so that it can be compared between samples. At the same time, the partial least squares method is used for feature extraction, which can extract potential information from the original data, reduce irrelevant noise, and reduce model complexity.

[0081] In addition, in order to solve the problem of data imbalance in different sample categories, the present invention divides the data at the patient level and uses the k-fold cross validation method for training. Each fold of data keeps the distribution consistency as much as possible and ensures that all categories are included in each fold of data. The purpose of this process is to use different subsets as training or test data each time, which can improve the reliability of the model in clinical diagnosis.

[0082] Step 3: Build a multi-task diagnosis model for oral cancer to simultaneously implement three tasks: tumor staging, lymph node staging, and histological grading. The overall network framework includes an embedding layer, a position encoding layer, a bidirectional Mamba module, a residual attention fusion module, layer normalization, and a multi-task classification head. The classification heads of different tasks are used to obtain their respective discrimination results.

[0083] Step 4: Train the constructed oral cancer multi-task diagnosis model.

[0084] The Pytorch deep learning framework was used for training in the environment of NVIDIA GeForce RTX 2080 Ti. The SGD optimizer was used to update the network weights, the momentum was set to 0.9, the weight decay parameter was set to 0.002, the initial learning rate was set to 0.05, and the cosine annealing algorithm was used as the learning rate adjustment strategy.

[0085] In addition, in order to ensure the credibility of the results, a 7-fold cross validation was performed, and each fold was trained for 300 rounds. The network uses Cross Entropy Loss as the loss function to train the network, and its calculation formula is as follows:

[0086] in, Indicates the true distribution The probability of the categories, Represents the first The loss function is used to measure the difference between the probability distribution predicted by the model and the true label. In addition, the softmax function is used to calculate the final category probability, and the category with the highest probability is taken as the final classification result.

[0087] Step 5: Predict the three tasks of tumor staging, lymph node staging, and histological grade. The model is evaluated by calculating evaluation indicators such as accuracy, precision, recall, and F1 score, and compared with the performance of the single-task model to prove the excellent performance of this model.

[0088] Specifically, multiple evaluation indicators are calculated based on the confusion matrix results of the three tasks to measure the performance of the model, including accuracy, precision, recall and F1 scores. Accuracy is the most intuitive performance indicator, which measures the proportion of the number of samples correctly predicted by the model to the total number of samples. Precision measures the proportion of the number of samples actually in this category among all samples predicted by the model for a certain category, and it focuses on how many of all samples predicted to be this category are correct. Recall is also called sensitivity, which measures how many of all samples that are actually a certain category are correctly predicted by the model, reflecting the ability of the model to capture samples of this category. The F1 score is an indicator used to measure the accuracy of the model, which can be regarded as a weighted average of the accuracy and recall of the model. Finally, the model evaluation results of this embodiment are compared with the results of other models to prove the excellent performance of this model.

[0089] Table 1 Evaluation index results of the model of the present invention and other models in the tasks of tumor staging, lymph node staging and histological grading Model Diagnostic tasks Accuracy (%) Accuracy (%) Recall (%) F1 Fraction (%) Tumor stage 85.3 ± 5.7 82.5 ± 3.1 82.8 ± 6.8 82.2 ± 7.0 Model of the present invention Lymph node staging 84.3 ± 6.4 84.4 ± 8.7 84.7 ± 7.5 83.3 ± 8.5 Histological grading 84.7 ± 5.6 83.5 ± 4.8 85.7 ± 5.7 85.0 ± 5.7 Tumor stage 81.5±3.8 82.1±3.3 80.2±4.1 81.1±3.5 Multi-Task Transformer Lymph node staging 81.3±4.7 83.0±4.3 80.1±4.9 81.5±4.4 Histological grading 83.0±3.3 83.3±4.2 76.7±3.7 80.2±2.8 Tumor stage 75.0±3.4 75.6±4.4 73.7±3.7 74.6±3.7 Multi-task AlexNet Lymph node staging 78.1±2.5 79.0±2.8 76.7±2.6 77.4±2.5 Histological grading 79.6±4.7 79.6±8.6 74.3±4.9 76.8±6.4 Tumor stage 72.2±5.1 71.9±6.4 70.6±5.1 71.3±5.7 Multi-task GoogleNet Lymph node staging 75.5±2.5 75.3±3.2 74.7±3.1 75.0±3.1 Histological grading 80.4±3.9 79.3±7.6 74.3±3.5 76.6±4.6 Tumor stage 69.9±3.2 70.4±5.7 66.5±3.1 68.3±3.6 Multi-task ResNet50 Lymph node staging 74.0±4.2 75.8±4.2 71.3±4.2 73.5±3.8 Histological grading 76.5±6.0 75.3±9.9 72.2±3.5 73.5±6.1 Tumor stage 81.7±8.0 81.7±8.5 81.9±9.0 80.3±9.3 Single-task Mamba Lymph node staging 80.6±9.1 78.4±12.3 81.6±10.5 78.0±11.9 Histological grading 85.3±4.8 80.8±12.1 82.8±9.7 81.4±10.9 It can be seen that the evaluation index results of the model of the present embodiment in the three tasks are better than the multi-task-AlexNet, multi-task-GoogleNet, multi-task-ResNet50 and multi-task-Transformer models, thanks to the Mamba network has a powerful characterization ability and reasoning ability comparable to the Transformer model, which uses structural parameter initialization to achieve stable optimization and superior computing performance, and the bidirectional Mamba module strengthens the model's ability to analyze Raman spectroscopy data, making the model of the present embodiment stand out. In addition, the results of the model of the present embodiment are compared with the single-task Mamba model, and it can be found that the results of the three tasks in the multi-task network are all improved, which can prove that the multi-task network has a greater advantage, which is that the multi-task model can share the underlying feature information from multiple related tasks, and the useful information between different tasks can be used to improve the performance of each task, which can make the model more comprehensively analyze the data and use its internal information to improve the discrimination ability of multiple tasks.

[0090] Embodiment 2 Based on the first embodiment, the second embodiment of the present invention provides an electronic device, including a memory and a processor, and a computer instruction stored in the memory and executed on the processor, and when the computer instruction is executed by the processor, the following steps are implemented: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

[0091] Embodiment 3 Based on the first embodiment, the third embodiment of the present invention provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the following steps are implemented: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model uses the residual attention fusion branch to extract local feature information in the Raman spectroscopy data, and generates multi-scale features by fusing the global feature information of the bidirectional Mamba branch.

[0092] Embodiment 4 Based on the first embodiment, the fourth embodiment of the present invention provides a computer program product, including a computer program / instruction, which implements the following steps when executed by a processor: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model uses the residual attention fusion branch to extract local feature information in the Raman spectroscopy data, and generates multi-scale features by fusing the global feature information of the bidirectional Mamba branch.

[0093] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-task oral cancer diagnosis system based on Raman spectroscopy, characterized in that: include: An acquisition module, used for acquiring Raman spectrum data; An oral cancer multi-task diagnosis module, used for processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

2. The oral cancer multi-task diagnosis system based on Raman spectroscopy according to claim 1, characterized in that: The bidirectional Mamba branch includes a position encoding layer and a plurality of bidirectional Mamba modules connected in sequence; The position encoding layer is used to add position encoding to the preprocessed Raman spectrum data, and the bidirectional Mamba module is used to perform bidirectional sequence global modeling on the input data.

3. The oral cancer multi-task diagnosis system based on Raman spectroscopy according to claim 2, characterized in that: The bidirectional sequence global modeling of the input data specifically includes: Perform layer normalization on the input data and linearly project them respectively to obtain the first sequence and the second sequence with the same dimension; Perform bidirectional SSM processing on the first sequence to obtain a forward sequence and a backward sequence, and fuse them with the second sequence processed by the activation function to obtain updated forward sequence and backward sequence; The updated forward sequence and backward sequence are added and processed by linear projection, and then connected with the input data residual to obtain the output sequence.

4. The oral cancer multi-task diagnosis system based on Raman spectroscopy according to claim 1, characterized in that: The residual attention fusion branch includes several residual attention fusion modules, and the residual attention fusion module is used to perform feature fusion on the same-layer output of the bidirectional Mamba branch and the output of the residual attention fusion module of the previous layer.

5. The oral cancer multi-task diagnosis system based on Raman spectroscopy according to claim 4, characterized in that: The feature fusion of the output of the same layer of the bidirectional Mamba branch and the output of the residual attention fusion module of the previous layer specifically includes: Perform layer normalization, convolution, and normalization operations on the same-layer outputs of the bidirectional Mamba branch in sequence to obtain a first output sequence; The output of the residual attention fusion module of the previous layer is layer-normalized and fused with the multi-head attention mechanism to obtain the second output sequence; The first output sequence and the second output sequence are fused, and convolution and normalization operations are performed in sequence, and residual connection is performed with the output of the first-level residual attention fusion module to obtain the fused output sequence.

6. The oral cancer multi-task diagnosis system based on Raman spectroscopy according to claim 1, characterized in that: The oral cancer multi-task diagnosis model also includes a multi-task classification head, which is a plurality of parallel multi-layer perceptrons.

7. The oral cancer multi-task diagnosis system based on Raman spectroscopy according to claim 1, characterized in that: The oral cancer multi-task diagnosis model was trained using the cross entropy loss function.

8. An electronic device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the following steps: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model includes a bidirectional Mamba branch and a residual attention fusion branch. The residual attention fusion branch is used to extract local feature information in the Raman spectral data, and the global feature information extracted by the bidirectional Mamba branch is fused to generate multi-scale features.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: The computer program / instructions, when executed by a processor, implement the following steps: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model uses the residual attention fusion branch to extract local feature information in the Raman spectroscopy data, and generates multi-scale features by fusing the global feature information of the bidirectional Mamba branch.

10. A computer program product comprising a computer program / instructions, characterized in that The computer program / instructions, when executed by a processor, implement the following steps: Used to obtain Raman spectroscopy data; Processing the Raman spectroscopy data through a trained oral cancer multi-task diagnosis model to generate and output tumor staging results, lymph node staging results, and histological staging results; The oral cancer multi-task diagnosis model uses the residual attention fusion branch to extract local feature information in the Raman spectroscopy data, and generates multi-scale features by fusing the global feature information of the bidirectional Mamba branch.

Images