Dose distribution optimization method based on brain-like calculation and multi-modal attention
By combining spiking neural networks and Transformer models, and utilizing multimodal attention and cross-attention mechanisms, the dose distribution of radiotherapy data is optimized, solving the problems of high model complexity and poor real-time performance in existing technologies, and achieving efficient and accurate radiation dose prediction and personalized treatment support.
Patent Information
- Application Number
- CN202411072782.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
AI Technical Summary
Existing technologies such as spiking neural networks (SNNs) and Transformer models suffer from problems such as high model complexity, poor unsupervised learning performance, strong data dependence, high computational complexity, difficulty in meeting real-time requirements, poor model interpretability, and strong hardware dependence when dealing with complex tasks.
By combining spiking neural networks and Transformer models, a cross-attention spiking network model is constructed through multimodal attention and cross-attention mechanisms. This model utilizes unlabeled radiotherapy data for unsupervised learning to optimize dose distribution.
It improves the accuracy and stability of radiation dose prediction, reduces computational complexity, enhances the model's generalization ability, supports the development of personalized treatment plans, and reduces patient risks and side effects.
Smart Images

Figure CN118940808A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of dose prediction, and in particular to a dose distribution optimization method based on brain-like computing and multimodal attention. Background Art
[0002] Spiking neural networks (SNNs) simulate biological neural systems and are known for their low power consumption and efficient computing. Unsupervised learning can automatically extract features from unlabeled data. The Transformer model, based on the self-attention mechanism, is widely used in fields such as natural language processing and computer vision. Nevertheless, both models have limitations when dealing with complex tasks: SNNs have limited applications in unsupervised learning and have poor results, while Transformers have high computational complexity when processing long sequences and are difficult to meet real-time requirements. Current challenges include:
[0003] 1. The combination of SNN and Transformer increases the complexity of the model and reduces training efficiency.
[0004] 2. Although unsupervised learning can improve generalization ability, the model may still overfit.
[0005] 3. Model performance is highly dependent on data preprocessing and feature extraction, and data quality has a significant impact.
[0006] 4. Complex models make it difficult to explain the decision-making process and lack transparency.
[0007] 5. SNN requires specific hardware support, such as dedicated chips, to achieve efficient computing and real-time processing.
[0008] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention
[0009] 1. Technical issues to be solved
[0010] In response to the shortcomings of the existing technology, the present invention provides a dose distribution optimization method based on brain-like computing and multimodal attention, which has the advantages of being able to better capture the key features and patterns in radiotherapy data and improve the accuracy of radiation dose prediction. It also solves the problem that Transformer has high computational complexity when processing long sequences and is difficult to meet real-time requirements.
[0011] (II) Technical solution
[0012] In order to achieve the above advantages of being able to better capture the key features and patterns in radiotherapy data and improve the accuracy of radiation dose prediction, the specific technical solutions adopted by the present invention are as follows:
[0013] A dose distribution optimization method based on brain-like computing and multimodal attention, the method comprising the following steps:
[0014] S1. Obtain historical dose distribution data and preprocess it, and use the preprocessed data as training data;
[0015] S2, constructing a pulse neural network model, and training the pulse neural network model using the training data;
[0016] S3, combining the trained spiking neural network model with the Transformer model with integrated attention mechanism to obtain a cross-attention spiking network model;
[0017] S4. Use the training data to train the cross-attention pulse network model, and use the trained cross-attention pulse network model to predict the dose distribution.
[0018] Furthermore, the acquisition of historical dose distribution data and preprocessing, and using the preprocessed data as training data includes the following steps:
[0019] S11, normalizing the acquired historical dose distribution data to obtain standard data;
[0020] S12. Use a Poisson coding function to convert the standard data into a format, and use the standard data after the format conversion as training data.
[0021] Furthermore, the expression of the Poisson coding function is:
[0022]
[0023] In the formula, encoded_data ij Indicates the encoded data; data ij represents standard data; k represents the parameter of Poisson coding.
[0024] Furthermore, the constructing of the spiking neural network model and training the spiking neural network model using training data comprises the following steps:
[0025] S21, establishing a spiking neural network structure, wherein the spiking neural network structure comprises an input layer, a hidden layer and an output layer, wherein the hidden layer uses spiking neurons;
[0026] S22, initializing the network parameters of the spiking neural network;
[0027] S23, converting the training data into a pulse sequence using a pulse sequence conversion equation;
[0028] S24. Based on the obtained pulse sequence, the pulse neural network model is trained using an unsupervised learning algorithm autoencoder.
[0029] Furthermore, the expression of the pulse sequence conversion equation is:
[0030]
[0031] Where p(t) represents the pulse sequence data; x(t) represents the training data; β represents the conversion coefficient, and θ represents the threshold.
[0032] Furthermore, the Transformer model with integrated attention mechanism includes a multi-head attention mechanism module and a cross attention mechanism module;
[0033] The multi-head attention mechanism module is used to perform attention operations on input data and merge the operation structures to obtain multi-head attention outputs;
[0034] The cross-attention mechanism module is used to perform a cross-attention operation on the multi-head attention output to obtain a cross-attention output.
[0035] Furthermore, performing the attention operation on the input data and merging the operation structures to obtain the multi-head attention output includes the following steps:
[0036] Obtain input data, and perform linear transformation processing on the input data through each head to obtain a first query vector, a first key vector, and a first value vector;
[0037] Calculate the attention scores of the first query vector and the first key vector of each head using the scaled dot product method;
[0038] Perform Softmax normalization on the obtained attention scores to obtain the first probability distribution;
[0039] Based on the obtained first probability distribution, weighted sum is performed on the first value vector of each head to obtain the output of each head;
[0040] The outputs of all heads are merged and linearly transformed again to obtain the multi-head attention output.
[0041] Further, performing a cross attention operation on the multi-head attention output to obtain the cross attention output includes the following steps:
[0042] Convert the obtained multi-head attention output into a second key vector, a second query vector, and a second value vector through matrix multiplication;
[0043] Calculate a cross-attention score using the second query vector and the second key vector;
[0044] Perform Softmax normalization on the obtained cross attention scores to obtain the second probability distribution;
[0045] Based on the obtained second probability distribution, the second value vector is weighted summed to obtain the cross attention output.
[0046] Furthermore, the expression of the cross-attention pulse network model is:
[0047] Output=Transformer(SNN(X));
[0048] In the formula, Output represents the target sequence matrix; Transformer represents the deep learning model based on the self-attention mechanism; X represents the dose distribution data; SNN represents the spiking neural network model.
[0049] Furthermore, the loss function used in the training process of the cross-attention pulse network model is a combined loss function, and the expression of the combined loss function is:
[0050] L total =L SNN +λ·L AE ;
[0051] Where, L total represents the combined loss function; L SNN represents the original loss function of the spiking neural network; L AE represents the loss function of the unsupervised learning algorithm autoencoder; λ represents the regularization coefficient.
[0052] (III) Beneficial effects
[0053] Compared with the prior art, the present invention provides a dose distribution optimization method based on brain-like computing and multimodal attention, which has the following beneficial effects:
[0054] 1. The present invention improves model performance and reduces computational complexity through data processing. It can also utilize a large amount of unlabeled radiotherapy data to automatically learn feature representation through unsupervised learning of pulse neural networks, without the need for excessive data annotation, thus saving time and labor costs.
[0055] 2. The present invention combines a multi-head attention mechanism to improve the expressiveness of the model, which can better capture the key features and patterns in radiotherapy data and improve the accuracy of radiation dose prediction. It comprehensively utilizes the unsupervised learning of pulse neural networks and the multi-head attention mechanism to effectively enhance the feature learning and model expression capabilities. The combination of pulse neural networks and Transformer models achieves effective management of computational complexity and further improves the performance of the model.
[0056] 3. The present invention realizes information interaction between different sequences by introducing the cross-attention mechanism, which effectively promotes information interaction between different sequences. It can better process sequence information in radiotherapy data, improve the accuracy and stability of radiation dose prediction, has strong generalization ability, and is suitable for a variety of tasks.
[0057] 4. The present invention accurately predicts radiation doses, allowing doctors to better formulate personalized treatment plans, improve the accuracy and effectiveness of radiotherapy, and reduce risks and side effects for patients. The application of radiation dose prediction in radiotherapy is highly innovative and can bring more technical support and assistance to radiotherapy, which is expected to improve treatment effects and improve patients' survival rate and quality of life. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0059] Figure 1 is a flow chart of a dose distribution optimization method based on brain-like computing and multimodal attention according to an embodiment of the present invention;
[0060] Figure 2 3 is a schematic diagram of the application of a dose distribution optimization method based on brain-like computing and multimodal attention according to an embodiment of the present invention. DETAILED DESCRIPTION
[0061] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0062] According to an embodiment of the present invention, a dose distribution optimization method based on brain-like computing and multimodal attention is provided.
[0063] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1-2 As shown, according to the dose distribution optimization method based on brain-like computing and multimodal attention according to an embodiment of the present invention, the method includes the following steps:
[0064] S1. Obtain historical dose distribution data and preprocess it, and use the preprocessed data as training data;
[0065] It should be noted that historical dose distribution data include treatment plan dose distribution (the expected dose distribution calculated by the radiotherapy planning system) and actual dose distribution (the actual irradiation dose obtained by the dose verification system). The dose verification system includes dose verification images, dose measurement equipment, etc. There is no radiotherapy dose distribution data in medical images: dose refers to the radiation dose of radiotherapy. Medical images do not contain specific information about dose, and image analysis technology can be used to predict the dose distribution at the tumor location and its surroundings.
[0066] Preferably, the obtaining of historical dose distribution data and preprocessing the data, and using the preprocessed data as training data comprises the following steps:
[0067] S11, normalizing the acquired historical dose distribution data to obtain standard data;
[0068] It should be noted that the dose distribution data is normalized to ensure that the input data is on the same scale, which is convenient for subsequent autoencoder and SNN network (spiking neural network) processing. Normalization can speed up the convergence of the network and improve the stability of the model.
[0069] S12. Use a Poisson coding function to convert the standard data into a format, and use the standard data after the format conversion as training data.
[0070] It should be noted that Poisson encoding is used to convert the original data into a format that can be processed by the neural network. The Poisson encoding function can be used to convert the input data (assuming it is a two-dimensional array with a shape of (batch_size, num_features) into a representation of the Poisson distribution.
[0071] Specifically, the expression of the Poisson coding function is:
[0072]
[0073] In the formula, encoded_data ij Indicates the encoded data; data ij represents standard data; k represents the parameter of Poisson coding.
[0074] S2, constructing a pulse neural network model, and training the pulse neural network model using the training data;
[0075] Preferably, the constructing of the spiking neural network model and training the spiking neural network model using training data comprises the following steps:
[0076] S21, establishing a spiking neural network structure, wherein the spiking neural network structure comprises an input layer, a hidden layer and an output layer, wherein the hidden layer uses spiking neurons;
[0077] It should be noted that the input layer is used to receive raw data. The hidden layer uses pulse neurons, and the state updates of these neurons follow a specific differential equation. The expression of the differential equation is:
[0078]
[0079] In the formula, V represents the neuron voltage, τ represents the time constant, and w i represents the connection weight, s i (t) represents the input pulse train. dV represents the differential of voltage V with respect to time, and dV / dt reflects the rate of change of voltage v with time t.
[0080] Where V is voltage, which represents the state of the neuron in SNN.
[0081] τ: Time constant that controls the rate at which neurons fire.
[0082] w i : Weight, the weight coefficient connecting different neurons, determines the impact of the input on the activation state of the neuron.
[0083] s i (t): Pulse occurrence probability, which means the probability of the i-th neuron generating a pulse at time t.
[0084] SNN: Spiking neural network can simulate biological nervous system and is used to simulate the pulse coding characteristics of biological system.
[0085] InputLayer: Input layer, receiving input data, input data X∈R n×d , where n represents the sample size, d represents the dimension of each sample, and R represents the input data set.
[0086] HiddenLayer: Hidden layer, the layer that implements data processing in the network.
[0087] OutputLayer: Output layer, which produces the final output of the network.
[0088] S22, initializing the network parameters of the spiking neural network;
[0089] It should be noted that the network parameters of the initialized spiking neural network are the weights and time constants of the initialized network.
[0090] S23, converting the training data into a pulse sequence using a pulse sequence conversion equation;
[0091] Specifically, the pulse sequence conversion equation is expressed as:
[0092]
[0093] Where p(t) represents the pulse sequence data, which is the probability of converting the input data into a pulse signal; x(t) represents the training data; β represents the conversion coefficient, which reflects the slope parameter of the nonlinear conversion from the input data to the probability of pulse occurrence; θ represents the threshold, which determines the threshold of pulse occurrence.
[0094] S24. Based on the obtained pulse sequence, the pulse neural network model is trained using an unsupervised learning algorithm autoencoder.
[0095] Specifically, the unsupervised learning algorithm is the contrastive divergence algorithm. Then the gradient descent method is used to train the SNN, and the weights of the autoencoder and SNN are adjusted at the same time: when training the SNN, not only the weight of the SNN is adjusted, but also the weight of the autoencoder is adjusted to ensure the effectiveness and accuracy of the entire model.
[0096] It should be noted that after the spiking neural network, an autoencoder is implemented to determine the number of layers of the encoder and decoder, the number of neurons in each layer, etc. It consists of an encoder and a decoder and is used for unsupervised learning, with the goal of learning a low-dimensional representation of the data.
[0097] S3. Combine the trained spiking neural network model with the Transformer model with integrated attention mechanism to obtain a cross-attention spiking neural network model; the spiking neural network model and the Transformer model are combined in a cascade connection. The Transformer model is a deep neural network based on the self-attention mechanism and has been widely used in fields such as natural language processing.
[0098] The output of the spiking neural network is used as the input of the Transformer model, and the Transformer model is used to process the output of the SNN. The integrated model module uses the output of the spiking neural network (SNN) as the input of the Transformer model, and uses the Transformer model to process the output of the SNN, thereby combining the advantages of the two models.
[0099] Combining the Spiking Neural Network with the Transformer Model, the SNN is combined with the Transformer model, where;
[0100] The input of SNN (Spiking Neural Network) usually refers to the original data, the dose distribution of the treatment plan, the dose distribution during the actual treatment, and contains the information that the model needs to learn and extract features.
[0101] SNN output: refers to the time pulse sequence matrix obtained after the spiking neural network processes the input data. The features of the input data are extracted and encoded. The output is obtained after SNN learning, and contains the abstract features of the input data.
[0102] Transformer input: refers to the output of SNN. The Transformer model receives the data processed by SNN as its input and further performs feature processing and pattern recognition.
[0103] Specifically, the expression of the cross-attention pulse network model is:
[0104] Output=Transformer(SNN(X));
[0105] In the formula, Output represents the target sequence matrix; Transformer represents the deep learning model based on the self-attention mechanism; X represents the dose distribution data; SNN represents the spiking neural network model.
[0106] It should be noted that before being integrated into the Transformer model, the weight matrices of multiple heads are defined and the attention weights are calculated to enhance the model's ability to focus on the input data. The multi-head attention mechanism is a core component of the Transformer model, which allows the model to focus on different parts of the input data in different subspaces in parallel.
[0107] The Transformer model with integrated attention mechanism includes a multi-head attention mechanism module and a cross attention mechanism module;
[0108] The multi-head attention mechanism module is used to perform attention operations on input data and merge the operation structures to obtain multi-head attention outputs; the multi-head attention mechanism includes multiple attention heads.
[0109] Among them, the cross attention mechanism allows information interaction between different sequences X and Y. Cross attention weight:
[0110]
[0111] X represents the dose distribution data of the treatment plan; Y represents the real dose value in the treatment, the actual dose achieved; Attention represents the attention mechanism, a mechanism that enables the model to focus on the important parts of the input data.
[0112] Q: Query, used to calculate the similarity with the key in the attention mechanism;
[0113] X T: Value, in the attention mechanism, the information output after weighting according to the similarity between the query and the key;
[0114] d k : The dimension of the key determines the complexity and capability of the attention mechanism;
[0115] Specifically, performing the attention operation on the input data and merging the operation structure to obtain the multi-head attention output includes the following steps:
[0116] Obtain input data, and perform linear transformation processing on the input data through each head to obtain a first query vector, a first key vector, and a first value vector;
[0117] Specifically, a linear transformation is applied to the input data to obtain query, key, and value vectors. A linear layer (fully connected layer) is used to transform the input data into query, key, and value vectors. This process involves learning the weight matrix W. q , W k and W v The linear transformation formula is as follows:
[0118] [query=x·W q ];
[0119] [key=x·W k ];
[0120] [value = x·W v ];
[0121] In the formula, x represents the input data, W q , W k and W v Represents the corresponding weight matrix.
[0122] Calculate the attention scores of the first query vector and the first key vector of each head using the scaled dot product method;
[0123] Specifically, the attention score is calculated for the query vector of each head and all key vectors, usually using a scaled dot product. For each attention head, the attention score of the query vector q and all key vectors k is calculated. Here, the scaled dot product attention mechanism is used. The formula for calculating the attention score is as follows:
[0124] [matmul_qk=q·k T ];
[0125]
[0126] [attention_weights=softmax(scaled_attention_logits)];
[0127] [output=attention_weights·v];
[0128] Where, d k represents the dimension of the key vector, softmax represents the softening function, which is used to convert the attention score into a probability distribution, and output represents the output of the attention mechanism. The shapes of the query vector q, key vector k, and value vector v are (batch_size, num_heads, seq_len, head_dim), where batch_size represents the batch size, num_heads represents the number of attention heads, seq_len represents the sequence length, and head_dim represents the dimension of each head. Scaling factor Usually taken as the square root of head_dim to avoid too large values in dot product calculations. matmul_qk represents the matrix multiplication operation of the query matrix (Q) and the key matrix (K), and the result is a new matrix that represents the similarity or association between the query and each key; scaled_attention_logits represents the score by which the result of matmul_qk is scaled before calculating the attention weights to avoid numerical instability when calculating softmax; attention_weights represents the attention weight between the input pulse input_pulse and a set of encoded pulses encoded_spikes.
[0129] Perform Softmax normalization on the obtained attention scores to obtain the first probability distribution;
[0130] Based on the obtained first probability distribution, weighted sum is performed on the first value vector of each head to obtain the output of each head;
[0131] The outputs of all heads are merged and linearly transformed again to obtain the multi-head attention output.
[0132] The cross attention mechanism module is used to perform a cross attention operation on the multi-head attention output to obtain a cross attention output. The cross attention mechanism realizes information interaction between different sequences.
[0133] It should be noted that the cross-attention mechanism module is used after the multi-head attention to enhance the model's understanding of the relationship between input data. The cross-attention weights are calculated to help the model understand the relationship between different input data. The cross-attention mechanism allows the model to pay attention to other inputs or contextual information while processing the current input, thereby enhancing the model's understanding of the relationship between input data.
[0134] Specifically, performing a cross attention operation on the multi-head attention output to obtain the cross attention output includes the following steps:
[0135] Convert the obtained multi-head attention output into a second key vector, a second query vector, and a second value vector through matrix multiplication;
[0136] The acquired context information (multi-head attention output) is converted into a key (second key vector), a query vector (second query vector) and a value vector (second value vector) through linear transformation (matrix multiplication). The implementation steps are:
[0137] The input context information is set to C, with a dimension of (m, n), where m is the amount of context information and n is the feature dimension.
[0138] Through the weight matrix W k , W q and W v Perform a linear transformation on C to obtain the key vector, query vector, and value vector:
[0139] K=C·W k ;
[0140] Q=C·W q ;
[0141] V=C·W v ;
[0142] Where W k , W q and W v is a learnable weight matrix with dimensions (n, d k )、(n,d q ) and (n,d v ), d k d q and d v are the dimensions of key, query, and value respectively. K, Q, and V represent the key vector, query vector, and value vector respectively.
[0143] Calculate a cross-attention score using the second query vector and the second key vector;
[0144] Specifically, the calculation formula for calculating the cross attention score is:
[0145] S=Q·K T ;
[0146] Where S represents the cross attention score, and its dimension is (n,n);
[0147] Perform Softmax normalization on the obtained cross attention scores to obtain the second probability distribution;
[0148] It should be noted that the calculation formula for Softmax normalization of the obtained cross attention score S is:
[0149]
[0150] Where Softmax(S) represents the second probability distribution, whose dimension is (m,m); S ij represents the denominator, which is the normalized probability of all input data sets having j elements, and the size of the attention weight assigned to the current position; k represents the kth S i (The previous pulse has been S i To avoid ambiguity); S ik represents the kth S i The sum of the exponentials of the normalized probability is 1; m means that both the row and column of the second probability distribution are m; S i represents the i-th cross attention score.
[0151]
[0152] Based on the obtained second probability distribution, the second value vector is weighted summed to obtain the cross attention output.
[0153] It should be noted that the weighted sum of the second value vector is used to obtain the calculation formula for the cross attention output:
[0154] O = Softmax(S)·V;
[0155] Where O represents the cross attention output, and its dimension is (m, d v ); V represents the value vector.
[0156] S4. Use the training data to train the cross-attention pulse network model, and use the trained cross-attention pulse network model to predict dose distribution.
[0157] In addition, after the model training is completed, the validation set and test set are used to evaluate the performance of the model. This step is an important part of testing the generalization ability of the model. The quality of the model is measured by evaluation indicators (such as accuracy, recall, etc.). According to the results of validation and testing, the model parameters are adjusted and optimized to improve the prediction accuracy and robustness of the model. This is an iterative process and may require multiple adjustments and tests until the model performance reaches the expected goal.
[0158] Among them, the trained model is used for prediction and the adaptive radiotherapy dose prediction results are output. According to the task requirements, the model effectively combines the advantages of unsupervised feature extraction, multi-head attention and cross-attention mechanisms, as well as the pulse neural network and Transformer models, thereby improving the performance and generalization ability of the model.
[0159] Specifically, the loss function used in the training process of the cross-attention pulse network model is a combined loss function, and the expression of the combined loss function is:
[0160] L total =L SNN +λ·L AE ;
[0161] Where, L total represents the combined loss function; L SNN represents the original loss function of the spiking neural network; L AE represents the loss function of the unsupervised learning algorithm autoencoder; λ represents the regularization coefficient.
[0162] It should be noted that after the model is integrated, it is necessary to train the model. By defining the loss function and then using the gradient descent method to optimize the model parameters, the model is trained to minimize the prediction error. Defining the loss function includes the following steps:
[0163] Step 1: Determine the original loss function L of SNN SNN ;
[0164] 1. Select the loss function type: Select the appropriate loss function according to the prediction task type of SNN (Product-based Neural Network). For regression tasks, you can choose mean square error (MSE) or root mean square error (RMSE); for classification tasks, you can choose cross-entropy loss (Cross-Entropy Loss).
[0165] 2. Define the loss function: According to the selected type, clarify the specific form of the loss function.
[0166] 1) For regression tasks, the loss function L SNN Defined as:
[0167]
[0168] Among them, y i is the true value, is the predicted value of SNN, and N is the number of samples.
[0169] 2) Loss function L of the autoencoder AE : For the autoencoder, L AE Defined as the difference between the input data and the reconstructed data, using MSE, L AE It can be defined as:
[0170]
[0171] Among them, x i is the original input data, is the data reconstructed by the autoencoder.
[0172] 3) Combined loss function;
[0173] Introducing the regularization coefficient λ: In order to balance the original loss of SNN and the loss of the autoencoder, a regularization coefficient λ is introduced. This coefficient is a hyperparameter that needs to be determined through experiments or cross-validation.
[0174] Combined loss function: The loss function of the autoencoder is added to the loss function of the SNN as a regularization term to form a total loss function:
[0175] L total =L SNN +λ·L AE ;
[0176] L SNN : It is the original loss function of SNN, which is used to measure the difference between the SNN predicted value and the true value and is a direct indicator of model performance.
[0177] L AE : is the loss function of the autoencoder, which is used to measure the difference between the autoencoder reconstructed data and the original data, reflecting the ability of the autoencoder to learn data representation. The loss function of the autoencoder is introduced as a regularization term, which is different from the previous loss function L SNN By adding L AE , which can prevent the model from overfitting and improve the model's generalization ability on unseen data;
[0178] λ: is the regularization coefficient, which is used to control the weight of the autoencoder loss in the total loss. By adjusting λ, a balance can be found between model complexity and generalization ability. The regularization coefficient λ plays a role in weighing L SNN and LAE Too large or too small will affect the overall performance of the model.
[0179] L total : is the total loss function. By combining the original loss of SNN and the loss of the autoencoder, it takes into account both the prediction performance of the model and the model's learning ability of data representation, which helps to improve the generalization ability of the model.
[0180] In addition, the combination of unsupervised learning of pulse neural networks and Transformer models, the integration of multi-head attention mechanisms and cross-attention mechanisms, and the comprehensive use of multiple key technologies provide new ideas and methods for sequence modeling and feature learning. This innovative design can promote the learning and expression capabilities of the model, and enrich the model's explanatory and generalization capabilities.
[0181] In summary, with the help of the above technical solutions of the present invention, the present invention improves the model performance and reduces the computational complexity through data processing, and can use a large amount of unlabeled radiotherapy data to automatically learn feature representation through unsupervised learning of pulse neural networks, without too much labeled data, saving time and labor costs. The present invention combines the multi-head attention mechanism to improve the expressiveness of the model, can better capture the key features and patterns in radiotherapy data, and improve the accuracy of radiation dose prediction. The comprehensive use of pulse neural network unsupervised learning and multi-head attention mechanism effectively improves the feature learning and model expression capabilities. The combination of pulse neural network and Transformer model realizes effective management of computational complexity and further improves the performance of the model. The present invention introduces the cross-attention mechanism to realize information interaction between different sequences, effectively promotes information interaction between different sequences, can better process sequence information in radiotherapy data, improves the accuracy and stability of radiation dose prediction, has strong generalization ability, and is suitable for a variety of tasks. By accurately predicting radiation doses, the present invention enables doctors to better formulate personalized treatment plans, improve the accuracy and effectiveness of radiotherapy, and reduce risks and side effects for patients. The application of radiation dose prediction in radiotherapy is highly innovative and can bring more technical support and assistance to radiotherapy, which is expected to improve treatment effects and improve patients' survival rate and quality of life.
[0182] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, optical storage, etc.) containing computer-usable program code.
[0183] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A dose distribution optimization method based on brain-like computing and multimodal attention, characterized in that: The method comprises the following steps: S1. Obtain historical dose distribution data and preprocess it, and use the preprocessed data as training data; S2, constructing a pulse neural network model, and training the pulse neural network model using the training data; S3, combining the trained spiking neural network model with the Transformer model with integrated attention mechanism to obtain a cross-attention spiking network model; S4. Use the training data to train the cross-attention pulse network model, and use the trained cross-attention pulse network model to predict the dose distribution.
2. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 1, characterized in that: The acquisition of historical dose distribution data and preprocessing, and using the preprocessed data as training data comprises the following steps: S11, normalizing the acquired historical dose distribution data to obtain standard data; S12. Use a Poisson coding function to convert the standard data into a format, and use the standard data after the format conversion as training data.
3. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 2, characterized in that: The expression of the Poisson coding function is: In the formula, encoded_data ij Indicates the encoded data; data ij represents standard data; k represents the parameter of Poisson coding.
4. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 1, characterized in that: The method of constructing a pulse neural network model and training the pulse neural network model using training data includes the following steps: S21, establishing a spiking neural network structure, wherein the spiking neural network structure comprises an input layer, a hidden layer and an output layer, wherein the hidden layer uses spiking neurons; S22, initializing the network parameters of the spiking neural network; S23, converting the training data into a pulse sequence using a pulse sequence conversion equation; S24. Based on the obtained pulse sequence, the pulse neural network model is trained using an unsupervised learning algorithm autoencoder.
5. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 4, characterized in that: The expression of the pulse sequence conversion equation is: Where p(t) represents the pulse sequence data; x(t) represents the training data; β represents the conversion coefficient, and θ represents the threshold.
6. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 1, characterized in that: The Transformer model with integrated attention mechanism includes a multi-head attention mechanism module and a cross attention mechanism module; The multi-head attention mechanism module is used to perform attention operations on input data and merge the operation structures to obtain multi-head attention outputs; The cross-attention mechanism module is used to perform a cross-attention operation on the multi-head attention output to obtain a cross-attention output.
7. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 6, characterized in that: The performing of attention operation on the input data and merging the operation structure to obtain the multi-head attention output comprises the following steps: Obtain input data, and perform linear transformation processing on the input data through each head to obtain a first query vector, a first key vector, and a first value vector; Calculate the attention scores of the first query vector and the first key vector of each head using the scaled dot product method; Perform Softmax normalization on the obtained attention scores to obtain the first probability distribution; Based on the obtained first probability distribution, weighted sum is performed on the first value vector of each head to obtain the output of each head; The outputs of all heads are merged and linearly transformed again to obtain the multi-head attention output.
8. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 6, characterized in that: The performing of the cross attention operation on the multi-head attention output to obtain the cross attention output comprises the following steps: Convert the obtained multi-head attention output into a second key vector, a second query vector, and a second value vector through matrix multiplication; Calculate a cross-attention score using the second query vector and the second key vector; Perform Softmax normalization on the obtained cross attention scores to obtain the second probability distribution; Based on the obtained second probability distribution, the second value vector is weighted summed to obtain the cross attention output.
9. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 1, characterized in that: The expression of the cross-attention pulse network model is: Output=Transformer(SNN(X)); In the formula, Output represents the target sequence matrix; Transformer represents the deep learning model based on the self-attention mechanism; X represents the dose distribution data; SNN represents the spiking neural network model.
10. The dose distribution optimization method based on brain-like computing and multimodal attention according to claim 1, characterized in that: The loss function used in the training process of the cross-attention pulse network model is a combined loss function, and the expression of the combined loss function is: L total =L SNN +λ·L AE ; Where, L total represents the combined loss function; L SNN represents the original loss function of the spiking neural network; L AE represents the loss function of the unsupervised learning algorithm autoencoder; λ represents the regularization coefficient.
Citation Information
Patent Citations
Tumor intensity modulated radiation therapy dose prediction method based on deep learning
CN114155934A
Radiotherapy dose prediction method and device based on coding and decoding network
CN117352128A
Cited By
Hyperspectral image classification method
CN119206379A
Network modeling method based on GNN and Transform model
CN119558346A
A network modeling method based on a GNN and a transformer model
CN119558346B
