A protein sequence feature representation method based on multi-scale modeling

Abstract

The application discloses a protein sequence feature representation method based on multi-scale modeling, which comprises the following steps: (1) passing a protein sequence through a ProtBert model to obtain a protein initial feature matrix; (2) using a multi-size sliding window on the protein initial feature matrix, respectively assigning attention coefficients to residues at different positions in the window, and calculating a protein attention feature matrix through an attention mechanism; and (3) respectively performing convolution on the protein attention feature matrix using multi-scale convolution kernels to finally obtain a protein sequence feature matrix based on the attention mechanism and the convolution network.

Description

Technical Field

[0001] This invention belongs to the field of protein bioinformatics computing, and particularly relates to a method for representing protein sequence features based on multi-scale modeling. Background Technology

[0002] Proteins are essential components of human cells, playing a crucial role in life processes through interactions with other cells, including DNA replication and transcription, enzyme catalysis, and gene expression regulation. Biological research on proteins typically employs methods such as cryo-electron microscopy to analyze their characteristics, but these methods are characterized by long experimental cycles and high costs.

[0003] In recent years, machine learning has been widely used in bioinformatics research. It can improve prediction accuracy in many applications, guide biological experiments, and has broader practical value.

[0004] Protein data is divided into sequence data and structural data. Sequence data consists of the order of amino acid residues in the protein, while structural data consists of the coordinate positions of atoms in the protein molecule. There are only 20 amino acids that make up proteins. For convenience, each amino acid is represented by a letter, such as "A" for alanine. Therefore, protein sequence data is a set of data composed of 20 letters.

[0005] However, the input data in machine learning or deep learning algorithms is usually in the form of numerical matrices. Currently, common methods for protein sequence feature extraction mainly use position-specific score matrices, one-hot encoding, or physicochemical properties, which are then concatenated to form a feature matrix. However, relying solely on this method is insufficient to uncover the deep features of proteins. To better utilize algorithms for protein research, it is necessary to propose a protein sequence feature representation method based on multi-scale modeling, which transforms protein sequence data into numerical matrices. Summary of the Invention

[0006] Purpose of the invention: To address the above problems, this invention proposes a protein sequence feature representation method based on multi-scale modeling. This method uses an attention mechanism to mine deeper features in protein sequence data and transforms the protein sequence data into a numerical matrix required for deep learning or machine learning.

[0007] Technical Solution: To achieve the objectives of this invention, this invention proposes a protein sequence feature representation method based on multi-scale modeling, which includes the following steps:

[0008] (1) The protein sequence information containing L residues is processed by the ProtBert model to obtain an initial protein feature matrix of L*1024 dimensions;

[0009] (2) Using sliding windows of sizes 5, 7 and 9 on the initial protein feature matrix, the attention mechanism algorithm is used to obtain the environment vector matrix. The environment vector matrices obtained from the three windows are horizontally concatenated to obtain the protein attention feature matrix.

[0010] (3) Convolve the protein attention feature matrix using one-dimensional convolution kernels with kernel sizes of 3, 5 and 7 respectively, and then concatenate the convolution feature matrices obtained by convolution kernels of different sizes laterally to obtain the protein sequence feature matrix based on the attention mechanism and convolution network.

[0011] Furthermore, in step (1), the protein sequence information containing L residues is processed through the ProtBert model to obtain an initial protein feature matrix of L*1024 dimensions. The method is as follows: After the protein sequence information containing L residues is processed through the ProtBert model, each residue will be represented by a 1*1024 dimension vector. The vectors of L residues are concatenated to obtain an initial protein feature matrix of L*1024 dimensions.

[0012] Furthermore, in step (2), an attention mechanism algorithm is used to obtain the environment vector matrix on the initial protein feature matrix using sliding windows of sizes 5, 7, and 9. The environment vector matrices obtained from the three windows are then horizontally concatenated to obtain the protein attention feature matrix. The method is as follows:

[0013] The initial feature matrix of the protein has a dimension of L*1024. Let the sliding window size be w, and let w take the values ​​of 5, 7 and 9 respectively. Then the feature matrix obtained in each sliding window has a dimension of w*1024.

[0014] Let the row vector located in the middle of the sliding window be the target residue vector r. i The remaining row vectors within the window are the vectors of adjacent residues r. j The adjacent residue vector is defined as follows:

[0015] When w takes the value 5, r j ∈{r i-2 ,r i-1 ,r i+1 ,r i+2} (1)

[0016] When w takes the value 7, r j ∈{r i-3 ,r i-2 ,r i-1 ,r i+1 ,r i+2 ,r i+3} (2)

[0017] When w takes the value 9, r j∈{r i-4 ,r i-3 ,r i-2 ,r i-1 ,r i+1 ,r i+2 ,r i+3 ,r i+4} (3)

[0018] When sliding windows of different sizes, ensure that each residue vector can become the target residue vector. Before the sliding window works, add (w-1) / 2 rows of zero vectors before and after the initial feature matrix of the protein.

[0019] When the sliding window starts working, each sliding window of a certain size uses the first row r1 of the initial protein feature matrix as the target residue vector, with a sliding step of 1. The second step uses the second row r2 of the initial protein feature matrix as the target residue vector, and so on, until the last step, when the sliding window uses r... L The target residue vector;

[0020] The similarity value between the target residue vector and each adjacent residue vector is calculated using the following formula:

[0021]

[0022] Perform SoftMax normalization to obtain the adjacent residue vector r j Attention coefficient α j :

[0023]

[0024] The attention coefficient α of all adjacent residues within the window j The vector r of the adjacent residue j Multiply and then sum to obtain the target residue vector r i environment vector g i The environment vector has a dimension of 1*1024, and the specific formula is as follows:

[0025] g i =∑α j r j (6)

[0026] After the sliding window completes its sliding, the environment vector matrix with the current sliding window size is obtained, which has a dimension of L*1024. The environment vector matrices obtained from the three sliding windows are horizontally concatenated to obtain the protein attention feature matrix, which has L rows and 3*1024, or 3072 columns, that is, the matrix dimension is L*3072.

[0027] Furthermore, in step (3), one-dimensional convolutional kernels with kernel sizes of 3, 5, and 7 are used to convolve the protein attention feature matrix, and the convolutional feature matrices obtained from convolutional kernels of different sizes are horizontally concatenated to finally obtain the protein sequence feature matrix based on the attention mechanism and convolutional network. The method is as follows:

[0028] The protein attention feature matrix has a dimension of L*3072, and is represented as S=[r′1,r′2,…,r′]. i ,…,r′ L-1 ,r′ L ] T , where r i ′ represents the residue vector in the protein attention feature matrix, with a dimension of 1*3072. Let the one-dimensional convolution kernel scale be k, that is, the convolution kernel dimension is 1*k, and k takes the value of 3, 5, or 7.

[0029] The three scales of one-dimensional convolution kernels respectively affect r i The convolution operation is performed using the following formula:

[0030]

[0031] The upper right subscript k is used to distinguish the scale of the current convolution kernel calculation, W k The weight matrix has the same dimensions as the convolution kernel scale, b k For the bias term, f k For activation function, This is a convolution-based representation of residue vectors. The dimension is 1*3072;

[0032] After each convolutional kernel performs a convolution on the protein attention feature matrix S, it generates a convolutional feature matrix with a dimension of L*3072. The convolutional feature matrices obtained from the three convolutional kernels of different sizes are horizontally concatenated to obtain the final protein sequence feature matrix β based on multi-scale modeling. Matrix β has L rows and 3*3072 columns, or 9216 columns, and the dimension of matrix β is L*9216. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the protein feature representation method in this invention;

[0034] Figure 2 This is a schematic diagram of a method for obtaining protein attention feature matrices;

[0035] Figure 3 This is a schematic diagram of obtaining protein sequence feature matrices based on multi-scale convolution kernels. Detailed Implementation

[0036] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] like Figure 1 As shown, this invention proposes a protein sequence feature representation method based on multi-scale modeling, which includes the following steps:

[0038] (1) The protein sequence information containing L residues is processed by the ProtBert model to obtain an initial protein feature matrix of L*1024 dimensions;

[0039] (2) Using sliding windows of sizes 5, 7 and 9 on the initial protein feature matrix, the attention mechanism algorithm is used to obtain the environment vector matrix. The environment vector matrices obtained from the three windows are horizontally concatenated to obtain the protein attention feature matrix.

[0040] (3) Convolve the protein attention feature matrix using one-dimensional convolution kernels with kernel sizes of 3, 5 and 7 respectively, and then concatenate the convolution feature matrices obtained by convolution kernels of different sizes laterally to obtain the protein sequence feature matrix based on the attention mechanism and convolution network.

[0041] Furthermore, in step (1), the protein sequence information containing L residues is processed through the ProtBert model to obtain an initial protein feature matrix of L*1024 dimensions. The method is as follows: After the protein sequence information containing L residues is processed through the ProtBert model, each residue will be represented by a 1*1024 dimension vector. The vectors of L residues are concatenated to obtain an initial protein feature matrix of L*1024 dimensions.

[0042] Furthermore, in step (2), an attention mechanism algorithm is used to obtain the environment vector matrix on the initial protein feature matrix using sliding windows of sizes 5, 7, and 9. The environment vector matrices obtained from the three windows are then horizontally concatenated to obtain the protein attention feature matrix. The method is as follows:

[0043] The initial feature matrix of the protein has a dimension of L*1024. Let the sliding window size be w, and let w take the values ​​of 5, 7 and 9 respectively. Then the feature matrix obtained in each sliding window has a dimension of w*1024.

[0044] Let the row vector located in the middle of the sliding window be the target residue vector r. i The remaining row vectors within the window are the vectors of adjacent residues r. j The adjacent residue vector is defined as follows:

[0045] When w takes the value 5, r j ∈{r i-2 ,r i-1 ,ri+1 ,r i+2} (1)

[0046] When w takes the value 7, r j ∈{r i-3 ,r i-2 ,r i-1 ,r i+1 ,r i+2 ,r i+3} (2)

[0047] When w takes the value 9, r j ∈{r i-4 ,r i-3 ,r i-2 ,r i-1 ,r i+1 ,r i+2 ,r i+3 ,r i+4} (3)

[0048] When sliding windows of different sizes, ensure that each residue vector can become the target residue vector. Before the sliding window works, add (w-1) / 2 rows of zero vectors before and after the initial feature matrix of the protein.

[0049] When the sliding window starts working, each sliding window of a certain size uses the first row r1 of the initial protein feature matrix as the target residue vector, with a sliding step of 1. The second step uses the second row r2 of the initial protein feature matrix as the target residue vector, and so on, until the last step, when the sliding window uses r... L The target residue vector;

[0050] The similarity value between the target residue vector and each adjacent residue vector is calculated using the following formula:

[0051]

[0052] Perform SoftMax normalization to obtain the adjacent residue vector r j Attention coefficient α j :

[0053]

[0054] The attention coefficient α of all adjacent residues within the window j The vector r of the adjacent residue j Multiply and then sum to obtain the target residue vector r i environment vector g i The environment vector has a dimension of 1*1024, and the specific formula is as follows:

[0055] g i =∑αj r j (6)

[0056] After the sliding window completes its sliding, the environment vector matrix with the current sliding window size is obtained, which has a dimension of L*1024. The environment vector matrices obtained from the three sliding windows are horizontally concatenated to obtain the protein attention feature matrix. The matrix has L rows and 3*1024 columns, which is 3072 columns, that is, the matrix dimension is L*3072.

[0057] Furthermore, in step (3), one-dimensional convolutional kernels with kernel sizes of 3, 5, and 7 are used to convolve the protein attention feature matrix, and the convolutional feature matrices obtained from convolutional kernels of different sizes are horizontally concatenated to finally obtain the protein sequence feature matrix based on the attention mechanism and convolutional network. The method is as follows:

[0058] The protein attention feature matrix has a dimension of L*3072, and is represented as S=[r′1,r′2,…,r′]. i ,…,r′ L-1 ,r′ L ] T , where r i ′ represents the residue vector in the protein attention feature matrix, with a dimension of 1*3072. Let the one-dimensional convolution kernel scale be k, that is, the convolution kernel dimension is 1*k, and k takes the value of 3, 5, or 7.

[0059] The three scales of one-dimensional convolution kernels respectively affect r i The convolution operation is performed using the following formula:

[0060]

[0061] The upper right subscript k is used to distinguish the scale of the current convolution kernel calculation, W k The weight matrix has the same dimensions as the convolution kernel scale, b k For the bias term, f k For activation function, This is a convolution-based representation of residue vectors. The dimension is 1*3072;

[0062] After each convolutional kernel performs a convolution on the protein attention feature matrix S, it generates a convolutional feature matrix with a dimension of L*3072. The convolutional feature matrices obtained from the three convolutional kernels of different sizes are horizontally concatenated to obtain the final protein sequence feature matrix β based on multi-scale modeling. Matrix β has L rows and 3*3072 columns, or 9216 columns, and the dimension of matrix β is L*9216.

Claims

1. A protein sequence feature representation method based on multi-scale modeling, characterized in that, The method includes the following steps: (1) The protein sequence information containing L residues is processed by the ProtBert model to obtain an initial protein feature matrix of L*1024 dimensions; (2) Using sliding windows of sizes 5, 7 and 9 on the initial protein feature matrix, the attention mechanism algorithm is used to obtain the environment vector matrix. The environment vector matrices obtained from the three windows are horizontally concatenated to obtain the protein attention feature matrix. (3) Convolve the protein attention feature matrix using one-dimensional convolution kernels with kernel sizes of 3, 5 and 7 respectively, and then concatenate the convolution feature matrices obtained by convolution kernels of different sizes laterally to obtain the protein sequence feature matrix based on the attention mechanism and convolution network. In step (2), an attention mechanism algorithm is used to obtain the environment vector matrix on the initial protein feature matrix using sliding windows of sizes 5, 7, and 9. The environment vector matrices obtained from the three windows are then horizontally concatenated to obtain the protein attention feature matrix. The method is as follows: The initial feature matrix of the protein has a dimension of L*1024. Let the sliding window size be w, and let w take the values ​​of 5, 7 and 9 respectively. Then the feature matrix obtained in each sliding window has a dimension of w*1024. Let the row vector located in the middle of the sliding window be the target residue vector r. i The remaining row vectors within the window are the vectors of adjacent residues r. j The adjacent residue vector is defined as follows: When w takes the value 5, r j ∈(r i-2 ,r i-1 ,r i+1, r i+2 } (1) When w takes the value 7, r j ∈{r i-3 ,r i-2 ,r i-1 ,r i+1 ,r i+2 ,r i+3 } (2) When w takes the value 9, r j ∈{r i-4 ,r i-3 ,r i-2 ,r i-1 ,r i+1 ,r i+2 ,r i+3 ,r i+4 } (3) When sliding windows of different sizes, ensure that each residue vector can become the target residue vector. Before the sliding window works, add (w-1) / 2 rows of zero vectors before and after the initial feature matrix of the protein. When the sliding window starts working, each sliding window of a certain size uses the first row r1 of the initial protein feature matrix as the target residue vector, with a sliding step of 1. The second step uses the second row r2 of the initial protein feature matrix as the target residue vector, and so on, until the last step, when the sliding window uses r... L The target residue vector; The similarity value between the target residue vector and each adjacent residue vector is calculated using the following formula: Perform SoftMax normalization to obtain the adjacent residue vector r j Attention coefficient α j : The attention coefficient α of all adjacent residues within the window j The vector r of the adjacent residue j Multiply and then sum to obtain the target residue vector r i environment vector g i The environment vector has a dimension of 1*1024, and the specific formula is as follows: g i ＝∑α j r j (6) After the sliding window completes its sliding, the environment vector matrix under the current sliding window size is obtained, with a dimension of L*1024. The environment vector matrices obtained from the three sliding windows are horizontally concatenated to obtain the protein attention feature matrix. The matrix has L rows and 3*1024 columns, that is, 3072 columns, that is, the matrix dimension is L*3072. In step (3), one-dimensional convolutional kernels with kernel sizes of 3, 5, and 7 are used to convolve the protein attention feature matrix. The convolutional feature matrices obtained from convolutional kernels of different sizes are then horizontally concatenated to finally obtain the protein sequence feature matrix based on the attention mechanism and convolutional network. The method is as follows: The protein attention feature matrix has a dimension of L*3072, and is represented as S=[r′1,r′2,…,r′]. i ,…,r′ L-1 ,r′ L ] T , where r′ i Let be the residue vector representation in the protein attention feature matrix, with a dimension of 1*3072. Let the one-dimensional convolution kernel scale be k, that is, the convolution kernel dimension is 1*k, and k takes the value of 3, 5, or 7. The three scales of one-dimensional convolution kernels respectively affect r′ i The formula for performing a convolution operation is as follows: The upper right subscript k is used to distinguish the scale of the current convolution kernel calculation, W k The weight matrix has the same dimensions as the convolution kernel scale, b k For the bias term, f k For activation function, This is a convolution-based representation of residue vectors. The dimension is 1*3072; After each convolutional kernel performs a convolution on the protein attention feature matrix S, it generates a convolutional feature matrix with a dimension of L*3072. The convolutional feature matrices obtained from the three convolutional kernels of different sizes are horizontally concatenated to obtain the final protein sequence feature matrix β based on multi-scale modeling. Matrix β has L rows and 3*3072 columns, or 9216 columns, and the dimension of matrix β is L*9216.

2. The protein sequence feature representation method based on multi-scale modeling according to claim 1, characterized in that, In step (1), the protein sequence information containing L residues is processed through the ProtBert model to obtain an initial protein feature matrix of L*1024 dimensions. The method is as follows: After the protein sequence information containing L residues is processed through the ProtBert model, each residue will be represented by a 1*1024 dimension vector. The vectors of L residues are concatenated to obtain an initial protein feature matrix of L*1024 dimensions.

Images