High-dimensional mass spectrum data feature selection method based on multi-scale self-attention mechanism

Abstract

The invention provides a high-dimensional mass spectrum data feature selection method based on a multi-scale self-attention mechanism. The method comprises the following steps: equally dividing high-dimensional mass spectrum data into m first feature subsets; processing the m first feature subsets by using a local multi-head self-attention mechanism model to obtain H attention weights of each feature; the local attention weight of each feature is obtained; processing the second feature subset by using a global multi-head self-attention mechanism model to obtain H attention weights of each aggregation feature; the global attention weight of each aggregation feature is obtained; based on the local attention weight of each feature and the global attention weight of the corresponding aggregation feature, determining the fusion attention weight of each feature; and determining the features of which the fusion attention weights are greater than a preset threshold as selected features, and generating dimension-reduced mass spectrum data based on the selected features. The expandability and the stability of the mass spectrum feature selection method are improved.

Description

Technical Field

[0001] The present application relates to the technical field of high-dimensional mass spectrometry data processing, and in particular to a high-dimensional mass spectrometry data feature selection method based on a multi-scale self-attention mechanism. Background Art

[0002] Feature selection technology is an important technology for processing high-dimensional mass spectrometry data. By performing feature selection on high-dimensional mass spectrometry data, the feature subset that contributes most to the classification or regression task can be selected from the complex features, thereby improving model performance, reducing computational costs, and enhancing interpretability, which helps researchers understand the entire experimental process and further analyze the experimental results.

[0003] There are three main common feature selection methods. Among them, the filtering method selects features based on the correlation between features and target variables or other statistical indicators, but lacks consideration of the correlation between features. The wrapper method regards feature selection as a search problem, using a certain search strategy to find the optimal or suboptimal feature subset, usually relying on a specific model to evaluate the quality of the feature subset. The embedded method performs feature selection and model training simultaneously, and selects the best feature combination by optimizing model performance. It has the advantages of efficient processing, strong interpretability, and reduced overfitting, but its selection range is limited by the available model types. Summary of the invention

[0004] In view of this, the present application provides a high-dimensional mass spectrometry data feature selection method based on a multi-scale self-attention mechanism to solve the above technical problems.

[0005] In a first aspect, an embodiment of the present application provides a method for selecting features of high-dimensional mass spectrometry data based on a multi-scale self-attention mechanism, comprising:

[0006] Acquire high-dimensional mass spectrometry data containing n features, and divide the n features into m first feature subsets;

[0007] The pre-trained local multi-head self-attention mechanism model containing H first self-attention heads is used to process the m first feature subsets respectively, and H attention weights of each feature in the first feature subset to which it belongs are obtained;

[0008] Calculate the average of the H attention weights of each feature in the first feature subset to which it belongs, and obtain the local attention weight of each feature after normalization;

[0009] Determine the average value of all features of each feature subset as an aggregate feature, and obtain a second feature subset containing m aggregate features;

[0010] The second feature subset is processed using a pre-trained global multi-head self-attention mechanism model containing H second self-attention heads to obtain H attention weights of each aggregated feature in the second feature subset;

[0011] Calculate the average of the H attention weights of each aggregated feature in the second feature subset, and obtain the global attention weight of each aggregated feature after normalization;

[0012] Determine the fused attention weight of each feature based on the local attention weight of each feature and the global attention weight of its corresponding aggregated feature;

[0013] Features whose fused attention weights are greater than a preset threshold are determined as selected features, and reduced-dimensional mass spectrometry data are generated based on the selected features.

[0014] In a possible implementation, a pre-trained local multi-head self-attention mechanism model including H first self-attention heads is used to process the m first feature subsets respectively, and H attention weights of each feature in the first feature subset to which it belongs are obtained, including:

[0015] The i-th first feature subset is represented as Among them, 1≤i≤m, is the first feature in the i-th first feature subset, is the second feature in the i-th first feature subset, is the n / mth feature in the i-th first feature subset;

[0016] The query parameter matrix of the hth first self-attention head based on the local multi-head self-attention mechanism model and key parameter matrix The i-th first feature subset As a vector of values Calculate the i-th first feature subset output by the h-th first self-attention head The attention weight vector S i,h , where the attention weight vector S i,h The dimensions and The dimension of the attention weight vector S i,h The jth element of The jth feature of The attention weight within .

[0017] In one possible implementation, a pre-trained global multi-head self-attention mechanism model including H second self-attention heads is used to process the second feature subset to obtain H attention weights of each aggregated feature in the second feature subset;

[0018] The second feature subset is represented as is the first aggregated feature in the second feature subset, is the second aggregate feature in the second feature subset, is the mth aggregate feature in the second feature subset;

[0019] The query parameter matrix of the h-th second self-attention head based on the global multi-head self-attention mechanism model and key parameter matrix The second feature subset As a vector of values Calculate the second feature subset output by the h-th second self-attention head The attention weight vector S′ is , where the dimension of the attention weight vector S′ is m, and the qth element of the attention weight vector S′ is The attention weight of the qth aggregated feature.

[0020] In one possible implementation, a fused attention weight of each feature is determined based on the local attention weight of each feature and the global attention weight of the corresponding aggregated feature;

[0021] Calculate the product of the local attention weight of the j-th feature and the global attention weight of the corresponding aggregated feature as the intermediate weight of the j-th feature;

[0022] The intermediate weights of all features are normalized based on the Softmax function to obtain the fused attention weight of each feature.

[0023] In one possible implementation, the method further includes: using a pre-trained classification network to process the reduced-dimensional mass spectrometry data to obtain a classification result of the high-dimensional mass spectrometry data; wherein the classification network includes a plurality of residual modules, a fully connected layer and a SoftMax function connected in sequence.

[0024] In one possible implementation, the method also includes: a step of jointly training a local multi-head self-attention mechanism model, a global multi-head self-attention mechanism model, and a classification network.

[0025] In one possible implementation, the steps of jointly training the local multi-head self-attention mechanism model, the global multi-head self-attention mechanism model, and the classification network include:

[0026] Obtaining a training set, wherein the training set includes a plurality of high-dimensional mass spectrometry data samples including n feature samples and their corresponding true classification results;

[0027] Divide n characteristic samples of the high-dimensional mass spectrometry data sample into m first characteristic subset samples;

[0028] The local multi-head self-attention mechanism model is used to process the m first feature subset samples respectively, and H attention weight samples of each feature in the first feature subset samples to which it belongs are obtained;

[0029] Calculate the average value of the H attention weight samples of each feature sample in the first feature subset sample to which it belongs, and obtain the local attention weight sample of each feature sample after normalization;

[0030] Determine the average value of all feature samples of each feature subset sample as an aggregate feature sample, and obtain a second feature subset sample containing m aggregate feature samples;

[0031] The second feature subset samples are processed using the global multi-head self-attention mechanism model to obtain H attention weight samples of each aggregated feature sample in the second feature subset samples;

[0032] Calculate the average value of H attention weight samples of each aggregate feature sample in the second feature subset sample, and obtain the global attention weight sample of each aggregate feature sample after normalization;

[0033] Determine a fused attention weight sample for each feature sample based on the local attention weight sample of each feature sample and the global attention weight sample of the corresponding aggregated feature sample;

[0034] Determine the feature samples whose fused attention weight samples are greater than a preset threshold as selected feature samples, and generate dimensionality-reduced mass spectrometry data samples based on the selected feature samples;

[0035] The dimensionality-reduced mass spectrometry data samples are processed using a classification network to obtain the predicted classification results of the high-dimensional mass spectrometry data samples;

[0036] Based on the predicted classification results and true classification results of high-dimensional mass spectrometry data samples, the loss value is calculated using the cross entropy function;

[0037] Based on the loss value, update the query parameter matrix and key parameter matrix of the H first self-attention heads of the local multi-head self-attention mechanism model, the query parameter matrix and key parameter matrix of the H second self-attention heads of the global multi-head self-attention mechanism model, and the parameters of the classification network.

[0038] In a second aspect, the embodiment of the present application provides a high-dimensional mass spectrometry data feature selection device based on a multi-scale self-attention mechanism, comprising:

[0039] An acquisition unit, used for acquiring high-dimensional mass spectrometry data containing n features, and dividing the n features into m first feature subsets;

[0040] A local processing unit, used to process the m first feature subsets respectively using a pre-trained local multi-head self-attention mechanism model including H first self-attention heads, to obtain H attention weights of each feature in the first feature subset to which it belongs;

[0041] The first processing unit is used to calculate the average value of H attention weights of each feature in the first feature subset to which it belongs, and obtain the local attention weight of each feature after normalization;

[0042] A determination unit, used to determine the average value of all features of each feature subset as an aggregate feature, and obtain a second feature subset containing m aggregate features;

[0043] A global processing unit, used to process the second feature subset using a pre-trained global multi-head self-attention mechanism model including H second self-attention heads, to obtain H attention weights of each aggregated feature in the second feature subset;

[0044] The second processing unit is used to calculate the average value of H attention weights of each aggregated feature in the second feature subset, and obtain the global attention weight of each aggregated feature after normalization;

[0045] A fusion unit, configured to determine a fused attention weight for each feature based on the local attention weight of each feature and the global attention weight of the corresponding aggregated feature;

[0046] The selection unit is used to determine the features whose fusion attention weights are greater than a preset threshold as selected features, and generate reduced-dimensional mass spectrometry data based on the selected features.

[0047] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of the embodiment of the present application when executing the computer program.

[0048] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a method of the embodiment of the present application is implemented.

[0049] This application improves the scalability and stability of the mass spectrometry feature selection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flow chart of a method for selecting features of high-dimensional mass spectrometry data based on a multi-scale self-attention mechanism provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of a first self-attention head provided in an embodiment of the present application;

[0053] Figure 3 A schematic diagram of a global multi-head self-attention mechanism model provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of a classification network provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of high-dimensional mass spectrometry data provided in an embodiment of the present application;

[0056] Figure 6 A functional structure diagram of a high-dimensional mass spectrometry data feature selection device based on a multi-scale self-attention mechanism provided in an embodiment of the present application;

[0057] Figure 7 A functional structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0059] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0060] First, the design concept of the embodiments of the present application is briefly introduced.

[0061] High-dimensional mass spectrometry data has a wide range of applications in biology, chemical analysis, and clinical diagnosis. Mass spectrometry data classification algorithms play an important role in compound identification, proteomics research, disease diagnosis, and drug efficacy evaluation. High-dimensional mass spectrometry data usually contain thousands or even tens of thousands of dimensional features. At the same time, due to mass spectrometer measurement errors or sample contamination, there are also features in the mass spectrometry features that are irrelevant to classification or even interfere with the accuracy of mass spectrometry classification. Usually, the number of samples used in mass spectrometry classification algorithms is less than the mass spectrometry feature dimension, resulting in the "dimensionality disaster" problem. In this case, directly performing operations such as classification and regression on the data will face problems such as overfitting, high computational complexity, and poor interpretability.

[0062] In the actual processing of mass spectrometry spectra, mass spectrometry features are often not just composed of fixed feature dimensions or a single feature distribution form. Mass spectrometry features of different dimensions and distribution forms have different effects on the classification accuracy of mass spectrometry classification algorithms. The current general mass spectrometry feature selection algorithm has the disadvantages of ignoring the interaction between features and over-relying on specific models. In addition, when calculating the correlation between features in high-dimensional mass spectrometry feature selection, the calculation of all global features will result in high computational costs, while only considering the correlation between some local features will lead to the problem of missing information on the correlation between features.

[0063] To solve the above technical problems, this application proposes a new mass spectrum feature selection method based on the multi-scale self-attention mechanism. First, the mass spectrum features are divided into m feature groups with equal dimensions according to the feature dimensions. For a single feature group, the number of its "multi-head self-attention heads" is set to H. For each "self-attention head" h, the corresponding query vector Q is generated by multiplying the input feature with the matrix. h , key vector K h Sum value vector V h . Calculate the attention weights of a certain dimension feature in the group on H “self-attention heads” and other features in the group respectively and sum them up as the self-attention weight of the dimension feature.

[0064] On the basis of using the local multi-head self-attention mechanism to obtain the attention weights calculated within each mass spectrum feature group, the global multi-head attention mechanism is used to calculate the attention weights between groups. First, the features of each group are summed and averaged to obtain m-dimensional features. Then, for H "self-attention heads", H groups of Q and K vectors can be obtained. For the Q and K vectors of each feature, multiple attention weights are calculated respectively according to the calculation method of the single-head self-attention mechanism. Finally, the attention outputs of these multiple heads are weighted and summed to obtain the final output of the global multi-head attention mechanism. Through the fusion of the local multi-head self-attention mechanism and the global multi-head self-attention mechanism, the self-attention weight of each feature obtained contains the correlation information between it and the local features and the global features at the same time, and avoids the high computational cost of calculating the self-attention weights between each feature and all other features.

[0065] For the self-attention weight of each feature obtained, a threshold can be set as a judgment criterion based on its self-attention weight value. Features with self-attention weight values ​​greater than the threshold are retained, and those with self-attention weight values ​​less than the threshold are discarded, thereby realizing a customized selection process for high-dimensional mass spectrometry features.

[0066] The selected feature set can be sent to a classification model based on a one-dimensional residual network. The model is composed of multiple residual modules. The output dimension of the last fully connected layer is equal to the number of categories of mass spectrometry data. The output of the last fully connected layer is calculated by the SoftMax function to obtain the prediction probability of each category. During the training process, the parameters of the classification algorithm and the parameters of the feature selection module are optimized in a coordinated manner to achieve adaptive coupling and performance optimization of feature selection and mass spectrometry classification algorithms.

[0067] The advantages of this method are:

[0068] 1. Fully consider the complexity of high-dimensional mass spectrometry feature classification. High-dimensional features cause the "dimensionality disaster" problem for mass spectrometry classification models, and there are outlier features or irrelevant features in high-dimensional features, which affect the classification effect. By adopting a high-dimensional mass spectrometry data feature selection method with a multi-scale self-attention mechanism, the local-global fusion self-attention weight of each feature is calculated, and by setting the selection threshold, controllable selection of feature dimensions is achieved.

[0069] 2. While retaining the correlation information between a certain dimension feature and local features and global features, it avoids the high computational cost caused by directly calculating the correlation between two features.

[0070] 3. Coupling the feature selection module with the mass spectrometry classification model and co-optimizing their parameters can effectively avoid the performance mismatch problem between modules caused by directly splicing the feature selection module with the mass spectrometry classification model, realize the adaptive fusion of mass spectrometry feature selection and mass spectrometry classification algorithms, and improve the scalability and stability of the mass spectrometry feature selection method.

[0071] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below.

[0072] like Figure 1 As shown, the embodiment of the present application provides a high-dimensional mass spectrometry data feature selection method based on a multi-scale self-attention mechanism, including:

[0073] Step 101: obtaining high-dimensional mass spectrometry data containing n features, and dividing the n features into m first feature subsets;

[0074] Exemplarily, high-dimensional mass spectrometry data is represented as X mass ={x1,x2,…,x n}, where each element represents a one-dimensional feature, and its corresponding category label is y mass . X mass Divide them into m feature subsets in the order of their mass axis (mz), expressed as: n is divisible by m.

[0075] Step 102: using a pre-trained local multi-head self-attention mechanism model including H first self-attention heads to process the m first feature subsets respectively, and obtaining H attention weights of each feature in the first feature subset to which it belongs;

[0076] Step 103: Calculate the average value of H attention weights of each feature in the first feature subset to which it belongs, and obtain the local attention weight of each feature after normalization.

[0077] Exemplarily, the Softmax function is used to normalize the average of the H attention weights of each feature.

[0078] Step 104: Determine the average value of all features of each feature subset as an aggregate feature, and obtain a second feature subset containing m aggregate features;

[0079] Step 105: Process the second feature subset using a pre-trained global multi-head self-attention mechanism model including H second self-attention heads to obtain H attention weights of each aggregated feature in the second feature subset;

[0080] Step 106: Calculate the average value of H attention weights of each aggregated feature in the second feature subset, and obtain the global attention weight of each aggregated feature after normalization;

[0081] Exemplarily, the Softmax function is used to normalize the average of the H attention weights of each aggregated feature.

[0082] Step 107: Determine the fused attention weight of each feature based on the local attention weight of each feature and the global attention weight of the corresponding aggregated feature;

[0083] Step 108: Determine the features whose fused attention weights are greater than a preset threshold as selected features, and generate reduced-dimensional mass spectrometry data based on the selected features.

[0084] In some embodiments, step 102 specifically includes:

[0085] The i-th first feature subset is represented as Contains n / m features, where 1≤i≤m, is the first feature in the i-th first feature subset, is the second feature in the i-th first feature subset, is the n / mth feature in the i-th first feature subset;

[0086] Based on the local multi-head self-attention mechanism model, the hth first self-attention head (such as Figure 2 ) and key parameter matrix The dimensions are: (n / m)×(n / m); the i-th first feature subset As a vector of values Calculate the i-th first feature subset output by the h-th first self-attention head The attention weight vector S i,h :

[0087]

[0088] where d k Represents the dimensions of the key parameter matrix.

[0089] Among them, the attention weight vector S i,h The dimensions and The dimension of the attention weight vector S i,h The jth element of The jth feature of The attention weight within .

[0090] Referring to the above attention weight calculation process of the hth first self-attention head output, the self-attention weight output result sequence of H first self-attention heads is calculated: S i,1 ,S i,2 ,…,S i,H The sequence is summed up and the mean is calculated as the final attention weight of each feature in the group, which represents the correlation between a certain dimensional mass spectrometry feature and its adjacent features.

[0091] In some embodiments, step 104 specifically includes:

[0092] In order to obtain the correlation information between different groups of features that are far apart, for the m groups of feature subsets obtained by grouping, first add each feature subset and calculate the average value to obtain an m-dimensional second feature subset: is the first aggregated feature in the second feature subset, is the second aggregate feature in the second feature subset, is the mth aggregate feature in the second feature subset;

[0093] Based on the global multi-head self-attention mechanism model (such as Figure 3 The query parameter matrix of the h-th second self-attention head is shown in and key parameter matrix The second feature subset As a vector of values Calculate the second feature subset output by the h-th second self-attention head The attention weight vector S′=(S′ 1 ,S′ 2 ,…,S′ H ), where the dimension of the attention weight vector S′ is m, and the qth element of the attention weight vector S′ is The attention weight of the qth aggregated feature.

[0094] In some embodiments, step 106 specifically includes:

[0095] Calculate the product of the local attention weight of the j-th feature and the global attention weight of the corresponding aggregated feature as the intermediate weight of the j-th feature;

[0096] The intermediate weights of all features are normalized based on the Softmax function to obtain the fused attention weight of each feature.

[0097] In some embodiments, step 107 specifically includes:

[0098] According to the set threshold p, the self-attention weights of all features are subtracted from p and divided by the absolute value of the difference. The result is processed by the ReLU function to obtain a feature selection vector, in which the corresponding value of the unselected feature is 0 and the corresponding value of the selected feature is 1.

[0099] The original mass spectrum feature is point-multiplied with the feature selection vector, and the features whose product is 0 are discarded to obtain the selected mass spectrum feature vector, thus achieving dimensionality reduction of the mass spectrum feature.

[0100] In some embodiments, the method further comprises: processing the reduced-dimensional mass spectrometry data using a pre-trained classification network to obtain a classification result of the high-dimensional mass spectrometry data; wherein the classification network comprises a plurality of residual modules, a fully connected layer, and a SoftMax function (such as Figure 4 shown).

[0101] In some embodiments, the method further includes: a step of jointly training a local multi-head self-attention mechanism model, a global multi-head self-attention mechanism model, and a classification network; specifically including:

[0102] Obtaining a training set, wherein the training set includes a plurality of high-dimensional mass spectrometry data samples including n feature samples and their corresponding true classification results;

[0103] Divide n characteristic samples of the high-dimensional mass spectrometry data sample into m first characteristic subset samples;

[0104] The local multi-head self-attention mechanism model is used to process the m first feature subset samples respectively, and H attention weight samples of each feature in the first feature subset samples to which it belongs are obtained;

[0105] Calculate the average value of the H attention weight samples of each feature sample in the first feature subset sample to which it belongs, and obtain the local attention weight sample of each feature sample after normalization;

[0106] Determine the average value of all feature samples of each feature subset sample as an aggregate feature sample, and obtain a second feature subset sample containing m aggregate feature samples;

[0107] The second feature subset samples are processed using the global multi-head self-attention mechanism model to obtain H attention weight samples of each aggregated feature sample in the second feature subset samples;

[0108] Calculate the average value of H attention weight samples of each aggregate feature sample in the second feature subset sample, and obtain the global attention weight sample of each aggregate feature sample after normalization;

[0109] Determine a fused attention weight sample for each feature sample based on the local attention weight sample of each feature sample and the global attention weight sample of the corresponding aggregated feature sample;

[0110] Determine the feature samples whose fused attention weight samples are greater than a preset threshold as selected feature samples, and generate dimensionality-reduced mass spectrometry data samples based on the selected feature samples;

[0111] The dimensionality-reduced mass spectrometry data samples are processed using a classification network to obtain the predicted classification results of the high-dimensional mass spectrometry data samples;

[0112] Based on the predicted classification results and true classification results of high-dimensional mass spectrometry data samples, the loss value is calculated using the cross entropy function;

[0113] Based on the loss value, update the query parameter matrix and key parameter matrix of the H first self-attention heads of the local multi-head self-attention mechanism model, the query parameter matrix and key parameter matrix of the H second self-attention heads of the global multi-head self-attention mechanism model, and the parameters of the classification network.

[0114] The specific implementation process of the method of this embodiment is described below using the mass spectrometry data collected from colon cancer patients as an example.

[0115] A piece of raw mass spectrometry data contains 13,500 dimensional features, such as Figure 6 As shown, the horizontal axis represents the mass-to-charge ratio (m / z), each feature corresponds to an m / z position, and the vertical axis represents the mass spectrum feature intensity value at each m / z position. The overall mass spectrum data contains three categories, namely healthy type, early colon cancer and advanced colon cancer.

[0116] First, the original features are divided into 10 feature subsets, and each feature subset contains 1350-dimensional features. The feature range of each feature subset is as follows:

[0117] Table 1

[0118] Feature subset Starting feature number End feature number Feature subset 1 1 1350 Feature subset 2 1351 2700 ... ... ... Feature subset 10 12151 13500

[0119] Local multi-head self-attention weight calculation:

[0120] For the i-th feature subset, the number of first self-attention heads is set to 5. For each first self-attention head, the self-attention weight value between each feature in the group and the remaining 1349-dimensional features is calculated. For each feature, a self-attention weight can be obtained through each "self-attention head", and a total of 5 self-attention weights can be obtained. The mean of the 5 self-attention weights of each feature is calculated respectively, and the mean of the 5 self-attention weights of each feature in the group is normalized based on the Softmax function to obtain the local normalized self-attention weight of each feature.

[0121] Global multi-head self-attention weight calculation:

[0122] The features within the 10 feature subsets are summed and averaged to obtain a 10-dimensional aggregate feature sequence. For the aggregate feature sequence, the number of second self-attention heads is also set to 5, and the self-attention weight value between each feature in the sequence and the remaining 9-dimensional features is calculated for each second self-attention head. Each aggregate feature can also obtain 5 self-attention weights. Each feature adds the 5 weights and normalizes them based on the Softmax function to obtain the normalized self-attention weight of each aggregate feature, which is the global self-attention weight.

[0123] Local-global self-attention weight fusion:

[0124] The 10 global self-attention weights corresponding to the above aggregated feature sequence are multiplied by the local normalized self-attention weights of the features in their respective groups, and all products are normalized based on the Softmax function to obtain the local-global self-attention fusion weight of each feature.

[0125] Mass Spectrum Feature Selection:

[0126] Set a threshold p, subtract p from the local-global self-attention fusion weights of all features, and divide by the absolute value of the difference to get a set of vectors consisting of 1 and -1. Use the ReLU function to process the vector to get the feature selection vector, where the corresponding value of the unselected feature is 0 and the corresponding value of the selected feature is 1. Multiply the feature selection vector with the original feature vector, eliminate the features whose product is 0, and get the selected mass spectrometry features.

[0127] One-dimensional residual network realizes mass spectrometry classification:

[0128] A one-dimensional residual network containing multiple residual modules was constructed as a mass spectrum classification network. The selected mass spectrum features were input into the network to obtain prediction results for three mass spectrum categories.

[0129] Based on the same inventive idea, the present application embodiment provides a high-dimensional mass spectrometry data feature selection device based on a multi-scale self-attention mechanism, see Figure 6 As shown, the high-dimensional mass spectrometry data feature selection device 200 based on the multi-scale self-attention mechanism provided in the embodiment of the present application at least includes:

[0130] An acquisition unit 201 is used to acquire high-dimensional mass spectrum data containing n features, and divide the n features into m first feature subsets;

[0131] A local processing unit 202 is used to process the m first feature subsets respectively using a pre-trained local multi-head self-attention mechanism model including H first self-attention heads to obtain H attention weights of each feature in the first feature subset to which it belongs;

[0132] The first processing unit 203 is used to calculate the average value of H attention weights of each feature in the first feature subset to which it belongs, and obtain the local attention weight of each feature after normalization processing;

[0133] A determining unit 204 is used to determine the average value of all features of each feature subset as an aggregate feature, and obtain a second feature subset including m aggregate features;

[0134] A global processing unit 205 is used to process the second feature subset using a pre-trained global multi-head self-attention mechanism model including H second self-attention heads to obtain H attention weights of each aggregated feature in the second feature subset;

[0135] The second processing unit 206 is used to calculate the average value of H attention weights of each aggregated feature in the second feature subset, and obtain the global attention weight of each aggregated feature after normalization;

[0136] A fusion unit 207, configured to determine a fused attention weight of each feature based on the local attention weight of each feature and the global attention weight of the corresponding aggregated feature;

[0137] The selection unit 208 is used to determine the features whose fusion attention weights are greater than a preset threshold as selected features, and generate reduced-dimensional mass spectrometry data based on the selected features.

[0138] It should be noted that the principle of solving the technical problem by the high-dimensional mass spectrometry data feature selection device 200 based on the multi-scale self-attention mechanism provided in the embodiment of the present application is similar to the method provided in the embodiment of the present application. Therefore, the implementation of the high-dimensional mass spectrometry data feature selection device 200 based on the multi-scale self-attention mechanism provided in the embodiment of the present application can refer to the implementation of the method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0139] Based on the above embodiments, the present application also provides an electronic device, referring to Figure 7 As shown, the electronic device 300 provided in the embodiment of the present application includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the high-dimensional mass spectrometry data feature selection method based on the multi-scale self-attention mechanism provided in the embodiment of the present application is implemented.

[0140] The electronic device 300 provided in the embodiment of the present application may further include a bus 303 connecting different components (including the processor 301 and the memory 302). The bus 303 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0141] The memory 302 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 3021 and / or a cache memory 3022 , and may further include a read-only memory (ROM) 3023 .

[0142] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, the program modules 3024 including but not limited to: an operating subsystem, one or more applications, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0143] The electronic device 300 may also communicate with one or more external devices 304 (e.g., keyboards, remote controls, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 300 (e.g., mobile phones, computers, etc.), and / or communicate with any device that enables the electronic device 300 to communicate with one or more other electronic devices 300 (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 305. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 306. Figure 7 As shown, the network adapter 306 communicates with other modules of the electronic device 300 via the bus 303. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0144] It should be noted that Figure 7 The electronic device 300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0145] The embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by the processor, the high-dimensional mass spectrometry data feature selection method based on the multi-scale self-attention mechanism provided in the embodiment of the present application is implemented. Specifically, the executable program can be built-in or installed in the electronic device 300, so that the electronic device 300 can implement the high-dimensional mass spectrometry data feature selection method based on the multi-scale self-attention mechanism provided in the embodiment of the present application by executing the built-in or installed executable program.

[0146] The high-dimensional mass spectrometry data feature selection method based on the multi-scale self-attention mechanism provided in the embodiment of the present application can also be implemented as a program product, which includes a program code. When the program product can be run on an electronic device 300, the program code is used to enable the electronic device 300 to execute the high-dimensional mass spectrometry data feature selection method based on the multi-scale self-attention mechanism provided in the embodiment of the present application.

[0147] The program product provided in the embodiments of the present application may adopt any combination of one or more readable media, wherein the readable medium may be a readable signal medium or a readable storage medium, and the readable storage medium may be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0148] The program product provided in the embodiment of the present application may adopt a CD-ROM and include program code, and may also be run on a computing device. However, the program product provided in the embodiment of the present application is not limited thereto. In the embodiment of the present application, the readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0149] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0150] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that any modification or equivalent replacement of the technical solution of the present application does not depart from the spirit and scope of the technical solution of the present application and should be included in the scope of the claims of the present application.

Claims

1. A high-dimensional mass spectrometry data feature selection method based on a multi-scale self-attention mechanism, characterized in that: include: Acquire high-dimensional mass spectrometry data containing n features, and divide the n features into m first feature subsets; The pre-trained local multi-head self-attention mechanism model containing H first self-attention heads is used to process the m first feature subsets respectively, and H attention weights of each feature in the first feature subset to which it belongs are obtained; Calculate the average of the H attention weights of each feature in the first feature subset to which it belongs, and obtain the local attention weight of each feature after normalization; Determine the average value of all features of each feature subset as an aggregate feature, and obtain a second feature subset containing m aggregate features; The second feature subset is processed using a pre-trained global multi-head self-attention mechanism model containing H second self-attention heads to obtain H attention weights of each aggregated feature in the second feature subset; Calculate the average of the H attention weights of each aggregated feature in the second feature subset, and obtain the global attention weight of each aggregated feature after normalization; Determine the fused attention weight of each feature based on the local attention weight of each feature and the global attention weight of its corresponding aggregated feature; Features whose fused attention weights are greater than a preset threshold are determined as selected features, and reduced-dimensional mass spectrometry data are generated based on the selected features.

2. The method according to claim 1, characterized in that The pre-trained local multi-head self-attention mechanism model containing H first self-attention heads is used to process the m first feature subsets respectively, and H attention weights of each feature in the first feature subset to which it belongs are obtained, including: The i-th first feature subset is represented as Among them, 1≤i≤m, is the first feature in the i-th first feature subset, is the second feature in the i-th first feature subset, is the n / mth feature in the i-th first feature subset; The query parameter matrix of the hth first self-attention head based on the local multi-head self-attention mechanism model and key parameter matrix The i-th first feature subset As a vector of values Calculate the i-th first feature subset output by the h-th first self-attention head The attention weight vector S i,h , where the attention weight vector S i,h The dimensions and The dimension of the attention weight vector S i,h The jth element of The jth feature of The attention weight within .

3. The method according to claim 2, characterized in that The second feature subset is processed using a pre-trained global multi-head self-attention mechanism model containing H second self-attention heads to obtain H attention weights for each aggregated feature in the second feature subset; The second feature subset is represented as is the first aggregated feature in the second feature subset, is the second aggregate feature in the second feature subset, is the mth aggregate feature in the second feature subset; The query parameter matrix of the h-th second self-attention head based on the global multi-head self-attention mechanism model and key parameter matrix The second feature subset As a vector of values Calculate the second feature subset output by the h-th second self-attention head The attention weight vector S ′ , where the attention weight vector S ′ The dimension is m, and the attention weight vector S ′ The qth element of The attention weight of the qth aggregated feature.

4. The method according to claim 3, characterized in that Determine the fused attention weight of each feature based on the local attention weight of each feature and the global attention weight of its corresponding aggregated feature; Calculate the product of the local attention weight of the j-th feature and the global attention weight of the corresponding aggregated feature as the intermediate weight of the j-th feature; The intermediate weights of all features are normalized based on the Softmax function to obtain the fused attention weight of each feature.

5. The method according to claim 4, characterized in that The method also includes: using a pre-trained classification network to process the reduced-dimensional mass spectrometry data to obtain a classification result of the high-dimensional mass spectrometry data; wherein the classification network includes a plurality of residual modules, a fully connected layer and a SoftMax function connected in sequence.

6. The method according to claim 5, characterized in that The method also includes: a step of jointly training a local multi-head self-attention mechanism model, a global multi-head self-attention mechanism model and a classification network.

7. The method according to claim 6, characterized in that The steps of jointly training the local multi-head self-attention mechanism model, the global multi-head self-attention mechanism model and the classification network include: Obtaining a training set, wherein the training set includes a plurality of high-dimensional mass spectrometry data samples including n feature samples and their corresponding true classification results; Divide n characteristic samples of the high-dimensional mass spectrometry data sample into m first characteristic subset samples; The local multi-head self-attention mechanism model is used to process the m first feature subset samples respectively, and H attention weight samples of each feature in the first feature subset samples to which it belongs are obtained; Calculate the average value of the H attention weight samples of each feature sample in the first feature subset sample to which it belongs, and obtain the local attention weight sample of each feature sample after normalization; Determine the average value of all feature samples of each feature subset sample as an aggregate feature sample, and obtain a second feature subset sample containing m aggregate feature samples; The second feature subset samples are processed using the global multi-head self-attention mechanism model to obtain H attention weight samples of each aggregated feature sample in the second feature subset samples; Calculate the average value of H attention weight samples of each aggregate feature sample in the second feature subset sample, and obtain the global attention weight sample of each aggregate feature sample after normalization; Determine a fused attention weight sample for each feature sample based on the local attention weight sample of each feature sample and the global attention weight sample of the corresponding aggregated feature sample; Determine the feature samples whose fused attention weight samples are greater than a preset threshold as selected feature samples, and generate dimensionality-reduced mass spectrometry data samples based on the selected feature samples; The dimensionality-reduced mass spectrometry data samples are processed using a classification network to obtain the predicted classification results of the high-dimensional mass spectrometry data samples; Based on the predicted classification results and true classification results of high-dimensional mass spectrometry data samples, the loss value is calculated using the cross entropy function; Based on the loss value, update the query parameter matrix and key parameter matrix of the H first self-attention heads of the local multi-head self-attention mechanism model, the query parameter matrix and key parameter matrix of the H second self-attention heads of the global multi-head self-attention mechanism model, and the parameters of the classification network.

8. A high-dimensional mass spectrometry data feature selection device based on a multi-scale self-attention mechanism, characterized in that: include: An acquisition unit, used for acquiring high-dimensional mass spectrometry data containing n features, and dividing the n features into m first feature subsets; A local processing unit, used to process the m first feature subsets respectively using a pre-trained local multi-head self-attention mechanism model including H first self-attention heads, to obtain H attention weights of each feature in the first feature subset to which it belongs; The first processing unit is used to calculate the average value of H attention weights of each feature in the first feature subset to which it belongs, and obtain the local attention weight of each feature after normalization; A determination unit, used to determine the average value of all features of each feature subset as an aggregate feature, and obtain a second feature subset containing m aggregate features; A global processing unit, used to process the second feature subset using a pre-trained global multi-head self-attention mechanism model including H second self-attention heads, to obtain H attention weights of each aggregated feature in the second feature subset; The second processing unit is used to calculate the average value of H attention weights of each aggregated feature in the second feature subset, and obtain the global attention weight of each aggregated feature after normalization; A fusion unit, configured to determine a fused attention weight for each feature based on the local attention weight of each feature and the global attention weight of the corresponding aggregated feature; The selection unit is used to determine the features whose fusion attention weights are greater than a preset threshold as selected features, and generate reduced-dimensional mass spectrometry data based on the selected features.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images