Radar HRRP target recognition method based on N2N and Bert

Through the radar HRRP target recognition method combined with N2N and Bert models, the problem of insufficient recognition rate in noise environment is solved. Through noise reduction and deep feature extraction, the attention mechanism and softmax classification are used to achieve more efficient radar target recognition.

CN112699782BActive Publication Date: 2024-12-03HANGZHOU KUNJUN TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011595694.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-29
Publication Date
2024-12-03
Estimated Expiration
2040-12-29

AI Technical Summary

Technical Problem

The existing radar HRRP target recognition method lacks recognition rate and robustness in a noisy environment, deep learning methods ignore timing information, and convolutional neural networks do not handle the sequence correlation of data sufficiently.

Method used

The radar HRRP target recognition method based on N2N and Bert is used to reduce the data through the N2N network, the convolutional neural network extracts local features, the Bert model extracts deep features, and uses attention mechanism and softmax for target recognition.

Benefits of technology

Effectively suppress noise, restore clean data, improve identification accuracy and robustness, make full use of the timing information and physical structure characteristics in HRRP data, and improve identification accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112699782B_ABST
    Figure CN112699782B_ABST
Patent Text Reader

Abstract

The invention discloses a radar HRRP target recognition method based on N2N and Bert, comprising the following steps: S1, collecting data and dividing it into a training set and a test set, sampling each category of the HRRP data collected by the radar, selecting a training set and a test set respectively, and then merging them to ensure that the data form of each category is sampled; S2, using an N2N module to reduce noise; S3, extracting the data passing through the N2N module through a convolutional neural network module, adjusting the number of channels, so as to input it into a Bert model; S4, using Bert to process the effective features extracted by CNN, and extracting deeper features; S5, classifying the features in a classification layer; S6, sending the HRRP test set processed by S1 to the trained model in step S5 for testing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of radar target recognition, and in particular relates to a radar HRRP target recognition method based on N2N and Bert. Background Art

[0002] One-dimensional high-resolution range profile (HRRP) is the echo of a target from a high-resolution broadband radar. The range resolution of a high-resolution broadband radar is much smaller than the target size, and its radar target echo signals often have the characteristics of diversity and sensitivity, making it difficult to capture and distinguish its features. In the field of radar target recognition, HRRP contains physical structure information that is extremely important for classification and recognition. Therefore, radar automatic target recognition methods based on HRRP have gradually become a hot topic in the field of radar automatic target recognition.

[0003] With the development of artificial intelligence, deep learning methods have gradually been applied to radar target recognition. Supervised learning-based methods can automatically extract the separable features of training data, which can be mainly divided into recurrent neural network deep learning methods, convolutional neural network deep learning methods and encoder-decoder methods. The first method uses the time domain segmentation method, which makes the features highly redundant and cannot efficiently utilize prior information. The latter two methods can effectively extract the overall envelope information of HRRP data, but ignore the sequence correlation and do not fully utilize the time series information in HRRP data.

[0004] Deep learning methods have greatly improved the accuracy and robustness of HRRP target recognition. However, in noisy environments, due to the large impact of noise on data, the recognition rate and robustness of many deep learning methods need to be improved. The HRRP target detection task in noisy environments needs further research. Summary of the invention

[0005] In view of the above technical problems, the present invention is used to provide a radar target recognition method based on N2N and Bert, and proposes a radar HRRP target recognition framework based on N2N and Bert. The data is first preprocessed to reduce the sensitivity, and the data is denoised through the N2N network. The local features and structural information are then extracted through the convolutional neural network. The deep features and temporal features are then extracted through Bert. Finally, the attention mechanism is adopted, and softmax is used for target recognition.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A radar HRRP target recognition based on N2N and Bert includes the following steps:

[0008] S1, collect data and divide it into training set and test set. The HRRP data collected by radar needs to be sampled for each category, and the training set and test set are selected separately, and then merged to ensure that the data form of each category is sampled. The ratio of training set and test set is kept at 7:3, and the training data set is recorded as D = {(x i ,y k )} i∈[1,n],k∈[1,c] , where x i represents the i-th sample, y k Indicates that the sample belongs to the kth category, a total of c categories of targets were collected, n represents the total number of samples, and data preprocessing is performed, including intensity normalization and center of gravity alignment;

[0009] S2, noise reduction using N2N module;

[0010] S3, the data after N2N module is extracted by convolutional neural network module and the number of channels is adjusted so as to be input into Bert model;

[0011] S4, uses Bert to process the effective features extracted by CNN and extract deeper features;

[0012] S5, classify the features at the classification layer;

[0013] S6, sending the HRRP test set processed by S1 to the model trained in step S5 for testing.

[0014] Preferably, in S1, data preprocessing is performed, and the specific steps are as follows:

[0015] S101, intensity normalization, assuming that the original HRRP data is represented by x raw =[x 1 ,x 2 ,x 3 …,x M ], where M represents the total number of distance units contained in HRRP, then the HRRP normalization calculation is as follows:

[0016]

[0017] S102, center of gravity alignment. The center of gravity alignment method translates the HRRP so that its center of gravity is located near the center point of the HRRP. The calculation method of the center of gravity g of the HRRP is as follows:

[0018]

[0019] Preferably, S2 uses an N2N module for noise reduction, and the specific steps are as follows:

[0020] S201, assuming the encoder input is After 7 convolution modules, except for the first and last convolution modules, there are no pooling layers. The middle 5 have pooling layers. The input and the output of the first four pooling layers are saved in the skip list for decoder superposition. The number of channels changes in the whole process is 1, 48, 48, 48, 48, 48;

[0021] S202, the decoder contains 11 convolution modules, among which the inputs of the 1st, 3rd, 5th, 7th, and 9th convolution modules are all the data after the output of the previous layer is dimensionally expanded and superimposed with the corresponding data in the skips, and then the number of channels is reduced to the original number of channels through the convolution layer. Each convolution module includes a convolution layer, a batch normalization layer, and an activation layer;

[0022] S203, convolution module, assuming the input is X, after N convolution kernels with kernel_size (1,5), the output is N represents the total number of channels, i represents the i-th channel;

[0023]

[0024] In the formula Represents the convolution operation. Batch Normalization calculates the mean and variance of the data in each mini_batch. Assuming there are N in a mini batch m HRRP samples, then the output is defined as where F n Represents the convolution output corresponding to the nth HRRP sample. In each small batch, The HRRP data in is batch normalized to obtain It is expressed as:

[0025]

[0026] Among them, F n (k,l) represents the lth element in the kth channel of the convolutional layer output corresponding to the HRRP sample before batch normalization. That is the HRRP data after batch normalization, α k and β k is the trainable parameter corresponding to the kth channel, ε is a very small number, in order to prevent the divisor from being 0, E(.) is the mean operation, Var(.) represents the variance operation;

[0027] Then use the activation function ReLU Each element in is nonlinearly activated to obtain If the input is Then the corresponding output after ReLU is It is expressed as:

[0028]

[0029] The pooling layer is mainly used to reduce the size of data and improve the robustness of features. The principle of the pooling layer is as follows:

[0030] Assuming the input data size is W×W, the convolution kernel size is F×F, the stride is S, and the padding is P, the following formula is used to obtain:

[0031]

[0032]

[0033] Preferably, the specific steps of S4 are as follows:

[0034] S401, determine the number of Bert layers to be used and construct a Bert model;

[0035] S402, assuming that the shape of the input X is [batch_size, M, N], batch_size is the number of batches, M is the dimension of each time point, N is the mapping sequence of each dimension after CNN extraction, that is, N-dimensional features, and add time series information, that is, position information, to it. According to the feature length, Bert will randomly initialize data of the same shape as the position information for learning during training;

[0036] S403, multi-head attention mechanism, extract effective features. The multi-head attention mechanism divides the N-dimensional features in the convolutional neural network into 12 groups, performs self-attention algorithm in each group, and then merges the results of the 12 groups. The output of the convolution block is X, and each group in X is multiplied by three randomly initialized matrices W q , W k , W V Get the three matrices Q, K, and V, and perform the following calculation to get Z i :

[0037]

[0038] Z i Represents the calculation results of each group, and then [Z 0 ...Z 11 ] Stack them up by column and multiply them by a randomly initialized W to get the final output Z of this layer;

[0039] S404, in the feed-forward layer, the N-dimensional features are mapped to high dimensions, and then mapped from high dimensions to low dimensions to further retain useful information;

[0040] S405, residual and layer normalization, the Add layer is the residual connection module;

[0041]

[0042] Where x∈R N×C×H×W , layer normalization calculates the mean and standard deviation of the data in the C, H, and W dimensions of each sample, retaining the N dimension, with μ as the mean, is the variance, α and β are parameters, and ε is a very small value;

[0043] S406, groups S402 to S405 together into one layer of Bert, with a total of 12 layers stacked.

[0044] Preferably, S5, classifying the features at the classification layer, the specific steps are as follows:

[0045] S501, after the Bert module, the output shape is [batch_size, M, N], and in the classifier, the attention mechanism is applied:

[0046]

[0047] α i is the weight of each dimension in the dimension M, L(i) is the feature of each dimension, and different weights are learned according to the importance of each dimension feature;

[0048] S502, through the softmax layer, the feature F ATT For classification, if the total number of targets contained in the training set is C, the test HRRP sample X test The probability corresponding to the i-th target in the target set is expressed as:

[0049]

[0050] Among them, exp(.) means exponential operation, F s (i) refers to the vector F s The i-th element in F s =W s F ATT , W s is the vector F s The weight matrix of the test HRRP sample X is test Classification to the maximum target probability c 0 middle:

[0051]

[0052] The present invention has the following beneficial effects:

[0053] (1) In the embodiment of the present invention, the N2N module is used to reduce noise on the HRRP data. The N2N module can effectively suppress noise and restore clean data. In addition, clean data is not required for training, and noisy data can be used for training.

[0054] (2) Convolutional neural network is applied in the embodiment of the present invention because the convolutional neural network can not only automatically extract features from HRRP samples, but also the sequence features extracted by the convolutional neural network retain the target structure information contained in the original echo, so as to facilitate the use of the subsequent Bert model.

[0055] (3) The Bert module is applied in the embodiment of the present invention. The Bert model can better utilize the prior of the overall fixed physical structure characteristics of the radar target, and the Bert network structure can make the model layer deeper, better abstract the characteristics of HRRP data, and perform better recognition.

[0056] (4) In the classification part of the embodiment of the present invention, an attention mechanism is used. Reclassification is to refine the features extracted by Bert, further screen out features that are more conducive to classification, and combine the features of different levels extracted by the Bert model to further improve the recognition accuracy of our model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is a flowchart of the steps of the radar HRRP target recognition method based on N2N and Bert according to an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] Reference Figure 1 , which is a flowchart of a radar HRRP target identification method based on N2N and Bert according to an embodiment of the present invention, comprising the following steps:

[0060] S1, the original HRRP data mainly has the problem of intensity sensitivity and translation sensitivity, and needs to be preprocessed. 2Normalize the intensity to improve the HRRP intensity sensitivity. The original HRRP data is intercepted from the radar echo data through the range window. When intercepting, the position of the recorded range image in the range gate is not fixed, which leads to the translation sensitivity of HRRP. Use the center of gravity alignment to process HRRP and reduce the translation sensitivity.

[0061] S2, input the above processed HRRP samples into the N2N module. The N2N module uses the encoding-decoding structure to learn clean data, reduce the impact of noise on subsequent processing steps, and improve the recognition rate of the classification model.

[0062] S3, input the above-processed HRRP samples into the convolutional neural network module. While retaining the target physical structure information contained in the HRRP sample, the convolutional neural network extracts and enhances the information contained in the HRRP local envelope. It improves the shortcomings of the sequence obtained by the time domain segmentation method, such as high redundancy, and the mutual constraints between the internal time point data dimension and the sequence length. The data that has passed the N2N module needs to be extracted by the convolutional neural network module and the number of channels adjusted so that it can be input into the Bert model.

[0063] S4, build the Bert model, adjust the hyperparameters, and optimize the model structure. In traditional models, the prediction of the RNN model can only rely on the information of the current moment and the previous moment. However, the HRRP data contains the overall structural prior of the target sample. It is not conducive to modeling and identifying it by only using the one-way information of the HRRP structure. Although the bidirectional RNN considers two directions, it is only a stack from left to right and from right to left. The current moment cannot make good use of the information of the previous and later moments.

[0064] The Bert module is used to abstract and describe the physical structure characteristics of the target contained in the HRRP, and integrate multi-level physical structure characteristics for identification. Compared with the traditional unidirectional RNN model and the bidirectional RNN model, the Bert model can better utilize the prior of the overall fixed physical structure characteristics of the radar target, making up for the shortcomings of the traditional RNN model and the bidirectional RNN. The attention mechanism in the Bert structure can better suppress features with weak correlation, give greater weight to important features, and reduce the interference of noise on both sides.

[0065] S5: Build a classifier to classify HRRP targets. For the output of Bert, use the idea of ​​attention mechanism again to retain more effective features, and finally use softmax classifier to classify the output of the network.

[0066] Testing phase:

[0067] S6, sending the HRRP test set processed by S1 to the model trained in steps S2, S3, S4, and S5 for testing.

[0068] In the specific application example, the detailed steps of S1 are:

[0069] S101, intensity normalization. Assume that the original HRRP data is represented by x raw =[x 1 ,x 2 ,x 3 …,x M ], where M represents the total number of distance units in the HRRP data, then the normalized HRRP can be expressed as:

[0070]

[0071] S1.2 Center of gravity alignment. In order to make training and testing have a unified standard, the center of gravity alignment method is used to eliminate translation sensitivity. The center of gravity alignment method translates the HRRP so that its center of gravity is near the center point of the HRRP. The calculation method of the center of gravity g is as follows:

[0072]

[0073] In the specific application example, S2 uses the N2N module for noise reduction. The specific steps are as follows:

[0074] S201, assuming the encoder input is After 7 convolution modules, except for the first and last convolution modules, there are no pooling layers, and the middle 5 have pooling layers. The input and the output of the first four pooling layers are saved in the skip list for the decoder superposition. The number of channels changes in the whole process is 1, 48, 48, 48, 48, 48, 48.

[0075] S202, the decoder contains 11 convolution modules, among which the inputs of the 1st, 3rd, 5th, 7th, and 9th convolution modules are all the data after the output of the previous layer is dimensionally expanded and superimposed with the corresponding data in the skips. Then, through the subsequent convolution layers, the number of channels is reduced to the original number of channels. Each convolution module basically includes a convolution layer, batch normalization, and activation layer, and some include a pooling layer.

[0076] S203, convolution module. Assume that the input is X, and after N convolution kernels with kernel_size (1,5), the output is N represents the total number of channels, which is N here, and i represents the i-th channel. Although the convolution kernel size is the same, the weight initialization is different, so these N channels are also different, and different local features can be extracted.

[0077]

[0078] In the formula Represents a convolution operation.

[0079] The data after the convolution layer needs to be further processed. In order to make the model easier to converge and the network training process more stable, batch normalization is added after convolution. It is mainly through calculating the mean and variance of the data in each small batch. Assuming that there are N in a small batch m HRRP samples, then the output is defined as where F n represents the convolution output corresponding to the nth HRRP sample. In each small batch, The HRRP data in is batch normalized to obtain It can be expressed as:

[0080]

[0081] Among them, F n (k,l) represents the lth element in the kth channel of the convolutional layer output corresponding to the HRRP sample before batch normalization. This is the HRRP data after batch normalization. k and β k is the trainable parameter corresponding to the kth channel, ε is a very small number to prevent the divisor from being 0. E(.) is the mean operation, and Var(.) represents the variance operation.

[0082] Then use the activation function ReLU Each element in is nonlinearly activated to obtain If the input is Then the corresponding output after ReLU is It can be expressed as:

[0083]

[0084] The pooling layer is mainly used to reduce the size of data and improve the robustness of features. The principle of the pooling layer is as follows:

[0085] Assuming the input data size is W×W, the convolution kernel size is F×F, the stride is S, and the padding is P, it can be obtained according to the following formula:

[0086]

[0087]

[0088] H is the height of the feature map after pooling, and W is the width.

[0089] In the specific application example, the detailed steps of S4 are:

[0090] S401, location information. The features extracted by the convolutional neural network module do not contain temporal information. The Bert module uses word embedding to dynamically learn location information.

[0091] S402, multi-head attention mechanism, extracting effective features. It can effectively utilize features of other dimensions, so that the distance between long-distance dependent features of the previous model is greatly shortened. The multi-head attention mechanism divides the N-dimensional features in the convolutional neural network into 12 groups, performs the self-attention algorithm in each group, and then merges the results of the 12 groups. Each dimensional feature in X is multiplied by three randomly initialized matrices W q , W k , W V We get three matrices Q, K, and V. We get Z by the following calculation:

[0092]

[0093] Then Z 0 to Z 11 Connect by columns and multiply by the randomly initialized matrix W to get the final Z.

[0094] S403, in the feedforward layer, the N-dimensional features are mapped to high dimensions, and then mapped from high dimensions to low dimensions to further retain effective features.

[0095] S404, residual and layer normalization, the residual layer is the residual connection module. Because deep neural networks need to be superimposed, the residual structure can effectively eliminate the information loss problem caused by the deepening of the number of layers, and can also quickly transmit the loss back to avoid gradient disappearance.

[0096] Like batch normalization in convolutional neural networks, layer normalization is added here to make the training process more stable.

[0097]

[0098] Where x∈R N×C×H×W , layer normalization calculates the mean and standard deviation of the data in dimensions C, H, and W for each sample, retaining the N dimension. μ is the mean, is the variance, α and β are parameters, and ε is a very small value.

[0099] In the specific application example, the detailed steps of S5 are:

[0100] S501, after the Bert module, the shape of the Output is [batch_size, 256, N]. In the classifier, the embodiment of the present invention applies the attention mechanism:

[0101]

[0102] α i In the dimension of 256, L(i) is the weight of each dimension, and L(i) is the feature of each dimension. Different weights are learned according to the importance of each dimensional feature.

[0103] S502, through the softmax layer feature F ATT If the total number of targets in the training set is C, the test HRRP sample X test The probability corresponding to the i-th target in the target set can be expressed as:

[0104]

[0105] Among them, exp(.) means exponential operation, F s (i) refers to the vector F s The i-th element in F s =W s F ATT , W s is the vector F s The weight matrix of HRRP sample X is tested by the maximum a posteriori probability. test Classification to the maximum target probability c 0 middle:

[0106]

[0107] S503, in the parameter training process of the proposed model, cross entropy is used as the cost function and mean square error. The cost function is used to calculate the loss during training and update the model parameters:

[0108]

[0109] Where N is the number of training samples in a batch, z(i) is used to represent the category of the nth training sample, and P(i|x train ) represents the probability value of the training sample corresponding to the i-th category. t(i) is the input of the i-th N2N, is the output of the i-th sample after the N2N module.

[0110] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.

Claims

1. A radar HRRP target recognition method based on N2N and Bert, characterized in that: The following steps are involved: S1, collect data and divide it into training set and test set. The HRRP data collected by radar needs to be sampled for each category, and the training set and test set are selected respectively, and then merged to ensure that the data form of each category is sampled; the ratio of training set and test set is kept at 7:3, and the training data set is recorded as D = {(x i ,y k )} i∈[1,n],k∈[1,c] , where x i represents the i-th sample, y k Indicates that the sample belongs to the kth category, a total of c categories of targets were collected, n represents the total number of samples, and data preprocessing is performed, including intensity normalization and center of gravity alignment; S2, noise reduction using N2N module; S3, the data after N2N module is extracted by convolutional neural network module and the number of channels is adjusted so as to be input into Bert model; S4, uses Bert to process the effective features extracted by CNN and extract deeper features; S5, classify the features at the classification layer; S6, sending the HRRP test set processed by S1 to the model trained in step S5 for testing; S5, classify the features at the classification layer. The specific steps are as follows: S501, after the Bert module, the output shape is [batch_size, M, N], and in the classifier, the attention mechanism is applied: a i is the weight of each dimension in the dimension M, L(i) is the feature of each dimension, and different weights are learned according to the importance of each dimension feature; S502, through the softmax layer feature F ATT For classification, if the total number of targets contained in the training set is C, the test HRRP sample X test The probability corresponding to the i-th target in the target set is expressed as: Among them, exp(.) means exponential operation, F s (i) refers to the vector F s The i-th element in F s =W s F ATT , W s is the vector F s The weight matrix of the test HRRP sample X is test Classify into the maximum target probability c0:

2. The radar HRRP target recognition method based on N2N and Bert as claimed in claim 1, characterized in that: In S1, data preprocessing is performed. The specific steps are as follows: S101, intensity normalization, assuming that the original HRRP data is represented by x raw =[x1,x2,x3…,x M ], where M represents the total number of distance units contained in HRRP, then the HRRP normalization calculation is as follows: S102, center of gravity alignment. The center of gravity alignment method translates the HRRP so that its center of gravity is located near the center point of the HRRP. The calculation method of the center of gravity g of the HRRP is as follows:

3. The radar HRRP target recognition method based on N2N and Bert as claimed in claim 1, characterized in that: S2, use the N2N module for noise reduction, the specific steps are as follows: S201, assuming the encoder input is After 7 convolution modules, except for the first and last convolution modules, there are no pooling layers. The middle 5 have pooling layers. The input and the output of the first four pooling layers are saved in the skip list for decoder superposition. The number of channels changes in the whole process is 1, 48, 48, 48, 48, 48; S202, the decoder contains 11 convolution modules, among which the inputs of the 1st, 3rd, 5th, 7th, and 9th convolution modules are all the data after the output of the previous layer is dimensionally expanded and superimposed with the corresponding data in the skips, and then the number of channels is reduced to the original number of channels through the convolution layer. Each convolution module includes a convolution layer, a batch normalization layer, and an activation layer; S203, convolution module, assuming the input is X, after N convolution kernels with kernel_size (1,5), the output is N represents the total number of channels, i represents the i-th channel; In the formula Represents the convolution operation, batch normalization Batch Normalization calculates the mean and variance of the data in each mini_batch, assuming that there are N in a mini batch m HRRP samples, then the output is defined as where F n Represents the convolution output corresponding to the nth HRRP sample. In each small batch, The HRRP data in is batch normalized to obtain It is expressed as: Among them, F n (k,l) represents the lth element in the kth channel of the convolutional layer output corresponding to the HRRP sample before batch normalization. That is the HRRP data after batch normalization, a k and β k is the trainable parameter corresponding to the kth channel, ε is a very small number, in order to prevent the divisor from being 0, E(.) is the mean operation, Var(.) represents the variance operation; Then use the activation function ReLU Each element in is nonlinearly activated to obtain If the input is Then the corresponding output after ReLU is It is expressed as: The pooling layer is mainly used to reduce the size of data and improve the robustness of features. The principle of the pooling layer is as follows: Assuming the input data size is W×W, the convolution kernel size is F×F, the stride is S, and the padding is P, the following formula is used to obtain:

4. The radar HRRP target recognition method based on N2N and Bert as claimed in claim 1, characterized in that: S4, use Bert to process the effective features extracted by CNN and extract deeper features. The specific steps are as follows: S401, determine the number of Bert layers to be used and construct a Bert model; S402, assuming that the shape of the input X is [batch_size, M, N], batch_size is the number of batches, M is the dimension of each time point, N is the mapping sequence of each dimension after CNN extraction, that is, N-dimensional features, and add time series information, that is, position information, to it. According to the feature length, Bert will randomly initialize data of the same shape as the position information for learning during training; S403, multi-head attention mechanism, extract effective features. The multi-head attention mechanism divides the N-dimensional features in the convolutional neural network into 12 groups, performs self-attention algorithm in each group, and then merges the results of the 12 groups. The output of the convolution block is X, and each group in X is multiplied by three randomly initialized matrices W q , W k , W V Get the three matrices Q, K, and V, and perform the following calculation to get Z i : Z i Indicates the calculation results of each group, and then [Z0...Z 11 ] Stack them up by column and multiply them by a randomly initialized W to get the final output Z of this layer; S404, in the feed-forward layer, the N-dimensional features are mapped to high dimensions, and then mapped from high dimensions to low dimensions to further retain useful information; S405, residual and layer normalization, the Add layer is the residual connection module; Where x i ∈R N×C×H×W , layer normalization calculates the mean and standard deviation of the data in the C, H, W dimensions of each sample, retaining the N dimension, μ L is the mean, is the variance, α and β are parameters, and ε is a very small value; S406, groups S402 to S405 together into one layer of Bert, with a total of 12 layers stacked.