A contrastive representation brain-machine video target detection method based on an attention mechanism

Abstract

The application discloses a contrastive representation brain-computer video target detection method based on an attention mechanism. The method uses two lightweight convolutional neural networks and a convolutional neural network with a multi-head attention mechanism to decode and classify the attention information of the brain, uses the attention mechanism of the electroencephalogram channel and the time dimension, and solves the class imbalance problem through a contrastive representation method. Specifically, in the training feature extractor stage, the attention mechanism is added to the feature extraction of the time and space dimensions, the information in the space and time can be explicitly decoupled, and the weak feature difference between the two classes is decoupled through the sample pair contrastive learning mode. In the later stage of feature extraction, the multi-head attention mechanism is introduced to learn multiple features of the sample to find the weak and small feature difference. Thus, whether the target appears and disappears and the time when the target appears and disappears can be located, and the P300 feature decay in long video detection and the long-tail distribution problem of positive and negative samples can be solved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer collaboration, and relates to a target detection method, in particular to a contrast representation brain-computer video target detection method based on an attention mechanism. BACKGROUND

[0002] A brain-computer interface can convert brain information into instructions to realize direct communication between the brain and external devices. In its application form, it includes helping to extend the motor ability of patients with diseases, such as a gaze-independent brain-computer interface (Gaze-independent BCI system) which can be used to restore the communication ability of patients with eye movement disorders. And using the powerful visual perception ability of people to cooperate with machines to realize intelligent enhancement, such as in the military field to find target images from a large number of pictures. In electroencephalogram target interpretation, a reliable way is to use the spatial attention mechanism of the brain, that is, to use the spatial distribution characteristics of the brain after event stimulation to judge the target characteristics. Image target interpretation includes target detection and positioning, and in most applications, the target source is a static picture, but there is no relevant solution for dynamic target determination in video streams and for target disappearance time node determination.

[0003] P300 is an event-related potential generated in the brain cognitive process, mainly related to expectation, intention, wakefulness, attention and other psychological factors. Sutton et al. found that when the human brain is stimulated by a small probability related event, a positive wave peak with a latency of about 300 ms will appear in the electroencephalogram signal, and P300 is thus named. In the research of P300-based iddball stimulation paradigm BCI system, the most classic application is the P300 character speller proposed and designed by Farwell and Donchin in 1988, which realizes character spelling through targeted stimulation. P300, as an important ERP feature, has been widely used in electroencephalogram research and application. In recent years, with the development of deep neural networks, more and more people use neural network technology to realize end-to-end feature extraction of electroencephalogram, and show superior performance. Since P300 is an electroencephalogram ERP feature triggered by a small probability event, the probability of target appearance is low in the experiment, which leads to an imbalance between the number of positive samples and the number of negative samples in the data collection process, that is, the problem of uneven sample quantity, also known as the long-tail distribution problem. In addition, due to the characteristics of P300, the P300 characteristics disappear when the target just appears, so that P300 cannot be used as a basis for determining whether the target appears in the later period of the target appearing video, and other features need to be found as the basis for judgment. It is still a great challenge to learn the potential invariants of electroencephalogram features. SUMMARY

[0004] In view of the deficiencies of the prior art, the application provides a contrastive representation brain-computer video target detection method based on an attention mechanism, which utilizes the attention mechanism of brain electrical channel and time dimension and a contrastive representation method to solve the class imbalance problem, decodes and classifies the attention information of the brain, thereby positioning whether the target of the subject's attention appears and at which time the target disappears after appearing, solving the problem that P300 features weaken in long video detection and other weak features need to be found, and the long-tail distribution problem of positive and negative samples.

[0005] A contrastive representation brain-computer video target detection method based on an attention mechanism, and the specific steps are as follows:

[0006] Step 1, obtaining spatial positioning electroencephalogram data

[0007] The electroencephalogram data of the subject when watching video content is collected by using the rapid serial visual presentation (RSVP) paradigm in the brain-computer interface (BCI). The time point and position of the target appearing in the video content are random, but the appearing time length is the same. The collected electroencephalogram data is divided into N samples C represents the number of electrodes of the electroencephalogram, and T represents the time length of one sample. Whether the target appears or not is taken as the class label of the sample, wherein the target is regarded as a positive sample, and the non-target is regarded as a negative sample.

[0008] Step 2, pairing of positive and negative samples

[0009] In order to solve the problem that the number of positive and negative samples does not match, the positive and negative samples are paired. Two samples of the same class are selected from the positive and negative samples to form a positive sample pair, and two samples of different classes are selected from the positive and negative samples by using a random selection method to construct a negative sample pair, and the repeated sample pairs are removed.

[0010] Step 3, spatial attention self-adaptive calibration based on electroencephalogram channel

[0011] For each input sample, the mutual dependence relationship between different electrode channels is modeled explicitly, the channel response characteristics are adaptively recalibrated, and the discriminative features in the spatial domain are explored. The specific method is to refer to ECAnet to perform squeezing and extraction operations on each electroencephalogram electrode channel of the sample. The squeezing operation is to first perform global average pooling on the sample, then input a 1x1 convolution layer with a convolution kernel size determined by an adaptive function, and compress the global time information of the sample into a group of channel descriptors z c , avoid dimension collapse, and capture the mutual dependence and interaction relationship between each electrode. After the channel descriptor is formed, a weight factor representing the weight of each channel electrode is generated through a gating mechanism and a sigmoid activation function. Finally, the weight factor is multiplied by the electroencephalogram electrode channel of the input sample to calibrate the channel attention.

[0012] Step 4, time channel based attention adaptive calibration

[0013] For the sample after channel attention calibration, the ECAnet is used to calculate the time channel in the sample in the same way as step 3, and the weight factor representing the weight of each time point is generated for the mutual dependence between different time dimensions, and the time attention is calibrated.

[0014] Step 5, channel feature depth adaptive extraction based on multi-head attention mechanism

[0015] The idea of multi-head attention mechanism is adopted to adaptively adjust the weight of channel attention, including separation, attention calculation and fusion. First, the sample data calibrated in step 4 is split into 3 identical branches, and then input into 3 SE Nets respectively, and different spatial feature directions are found respectively to generate different weights. Finally, the output results of the three SE Nets are fused by adding to extract deep channel features. In order to save the calculation amount, dimension reduction operation is adopted in each channel calculation, and in order to ensure the effectiveness of the data, the original data is directly copied to each attention calculation network by using the direct copying method.

[0016] Step 6, contrast representation learning

[0017] Two samples in a sample pair obtained in step 2 are sequentially processed by steps 3-5 to obtain feature vectors of the two samples in the mapping space, and then two fully connected layers are used to expand the feature vectors to calculate the cosine similarity or Euclidean distance as the distance of the two feature vectors in the sample mapping space. The distance between positive and negative sample pairs is used to return the loss, and then back propagation is performed to make the distance between two same samples as compact as possible and the distance between class samples as large as possible.

[0018] Step 7, target detection

[0019] After the reverse propagation training in step 6 is completed, the network parameters are fixed, the predicted category output by the classifier is compared with the true category of the sample, and the classifier is trained. Then input the sample with unknown category, judge whether there is target and the time of target disappearance.

[0020] The present application has the following beneficial effects:

[0021] The application adopts two lightweight convolutional neural networks and a convolutional neural network with a multi-head attention mechanism to effectively capture the channel and time characteristics of electroencephalogram. Compared with the existing RSVP target detection based on electroencephalogram, which is for static targets, the detection method proposed by the application is for the target paradigm appearing in the video, and for the P300 feature decay after the target appears in the video for a long time, the method of contrast learning is adopted to effectively find the inter-class feature difference. In the stage of training the feature extractor, attention mechanism is added to the extraction of time and space dimension features, which can explicitly decouple the spatial and temporal information, and through the intra-sample contrast learning method, the weak feature difference between the two classes is decoupled, and in the later stage of feature extraction, the multi-head attention mechanism is introduced to learn multiple features of the sample to find the weak and small feature difference. The sample pair method is also used to increase the number of sample pairs, solving the class imbalance problem. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A brain-computer video target detection method flow chart;

[0023] Figure 2 A paradigm video screenshot used in the embodiment;

[0024] Figure 3 A spatial attention adaptive calibration diagram based on electroencephalogram channels;

[0025] Figure 4 A time channel-based attention adaptive calibration diagram;

[0026] Figure 5 A channel feature depth adaptive extraction diagram based on a multi-head attention mechanism;

[0027] Figure 6 A contrast representation learning diagram;

[0028] Figure 7 A target detection diagram. DETAILED DESCRIPTION

[0029] The method of the application will be described in detail below in combination with the drawings. As shown in the drawings, a contrast representation brain-computer video target detection method based on an attention mechanism has the following specific steps: Figure 1

[0030] Step 1, obtaining spatial positioning electroencephalogram data

[0031] ​Before collecting EEG data, the experimental design must be carried out. This embodiment adopts the rapid serial visual presentation (RSVP) paradigm in the brain-computer interface (BCI). The stimulus video is an animation video with a total length of 10 seconds. The animation video is self-made through animation software. Each animation video will occasionally appear a target image in a certain position in the screen. Figure 2 As shown, the target image is a cartoon cow. The target image appears for a fixed 4-second duration in each animated video. The animated videos used in this example include seven categories: no target, target appearing at 0s, target appearing at 1s, target appearing at 2s, target appearing at 3s, target appearing at 4s, and target appearing at 5s. A total of 500 videos were played, with a 1:1 ratio of videos containing a target to videos without a target. During EEG data collection, participants were approximately 70cm from the screen playing the animated videos. Non-invasive EEG data were collected using a 64-channel Neuroscan device, employing a 10-20 electrode placement method. The impedance of all electrodes was maintained below 15Ω prior to acquisition, and the sampling rate was 1000 Hz. Participants were required to complete eight batches of viewing tasks, each containing 50 videos. After each video finished playing, a "+" icon appeared in the center of the screen for 2s, allowing participants to rest and focus. After completing each batch, participants could click the mouse to start watching the next batch or take a break. While watching the video, the subjects were required to pay attention to whether the target appeared and keep staring at the target after it appeared.

[0032] The collected EEG data was first processed with a Butterworth bandpass filter to 0.1-40 Hz. The 1000 Hz EEG data was then downsampled to 250 Hz. Finally, the continuous EEG data was divided into multiple segments, and the 300 ms pre-stimulus data was used to perform baseline calibration on the 0-1000 ms post-stimulus data segment. The EEG data without the target was then sliced ​​at regular intervals to serve as negative samples. The EEG data with the target was sliced ​​at 1-second intervals, with the preceding and following time periods distinguished as positive samples.

[0033] Step 2: Pairing positive and negative samples

[0034] For the pairing of positive and negative samples, first give an initial value NUM, select two samples of the same category from the positive samples and negative samples obtained in step 1 to form a positive sample pair, and construct right.

[0035] Then, two samples of different categories are selected from the positive samples and negative samples to form a negative sample pair, and a total of To avoid the problem of repeated sample pairs caused by random selection, the negative sample pairs are constructed as follows: given a threshold T, two initial values ​​I1 and I2 are selected each time, and a sliding window pairing is performed starting from the positions I1 and I2 in the positive and negative samples respectively. After reaching the threshold T, I1 and I2 are randomly selected again until the number of negative sample pairs reaches

[0036] Step 3: Channel-based spatial attention calibration

[0037] like Figure 3 As shown, for each sample First, it is converted from the form [m,n,c,t] to [m,c,n,t], where C represents the number of electrodes, T represents the length of a sample, N represents the total number of samples, m represents the number of samples in a batch, n represents the number of feature channels, c represents the number of electrode channels, and t represents the time node. By explicitly modeling the interdependencies between different channels, the electrode channel response characteristics are adaptively recalibrated to explore discriminative features in the spatial domain. Therefore, each converted sample is input into ECANet for squeezing and extraction operations. Unlike SENet, ECANet uses one-dimensional convolution operations to capture cross-channel dependencies. Using fully connected layers to capture relationships between all channels is not particularly efficient, especially for time series. A time slice may only have connections with adjacent time slices, and such connections may be unnecessary for time series.

[0038] The squeezing operation aggregates the feature maps in the spatial dimension of the input sample to generate a channel descriptor. The channel descriptor can be used to generate a global distribution embedding of the channel feature response and allow all layers to use information from the global receptive field of the network. The generation method is to perform a global average pooling operation on the sample:

[0039]

[0040] Among them, F sq () represents the global average pooling operation, z ic represents the descriptor of the cth channel in the i-th sample, Q ic Represents the signal of the cth channel in the i-th sample q ic (j) represents the jth time point of the cth channel in the i-th sample. Then, in order to explore the correlation between electrode channels, z ic Input a 1×1 convolution layer with a convolution kernel size of kernel to compress the global temporal information of the sample into a set of channel descriptors z cWhere kernel is calculated by the following adaptive function:

[0041]

[0042] Where abs() represents the calculation of absolute value, y and b are numerical fixed hyperparameters, and in this embodiment, y = 2, b = 1.

[0043] Then through a sigmoid activation function, a dimension increasing layer, and returning to the same channel number as the input, the channel descriptor z c is mapped to a set of channel weights as self-attention weights on the electrode channels to generate the activation factor s:

[0044] s = F ex (z i ,W) = σ(g(z i ,W))

[0045] F ex () represents a full connection operation for learning the correlation between channels, σ() represents a sigmoid activation function, g() represents a pooling layer and a one-dimensional adaptive convolution function, W ∈ R is a parameter of a one-dimensional convolution layer, z i represents the descriptor corresponding to the i-th sample, and finally the activation factor s is multiplied by the electroencephalogram electrode channel of the input sample to calibrate the channel attention, and the calibrated sample data is Where F scale (q ic ,s ic ) means that the weight s ic and the feature value q ic perform element multiplication based on the channel, and after the above operation, the data form is transposed from [m, c, n, t] back to [m, n, c, t].

[0046] Step 4, time channel-based attention adaptive calibration

[0047] As Figure 4 shown, for the sample after channel attention calibration , first convert it from the form [m, n, c, t] to the form [m, t, n, c], and then compress and extract in the same way as step 3, and generate a weight factor representing the weight of each time point for the mutual dependence between different time dimensions, calibrate the time attention, and after the operation, the data form is transposed from [m, c, n, t] back to [m, n, c, t] to obtain the output of this step is an output data finally obtained after attention calculation through two channels of space and time, which can enable the network to effectively extract fine-grained time and space features in the sample, and enable the network to pay more attention to those distinguishing features.

[0048] Step 5, channel feature depth self-adaptive extraction based on multi-head attention mechanism

[0049] The idea of multi-head attention mechanism is adopted to capture different feature behaviors in the feature space by using the same model, and to adaptively adjust the weight of channel attention, including separation, attention calculation and fusion.

[0050] The main idea of this step is to improve the fault tolerance of the network attention mechanism and to discover more features. In the same attention mechanism, if the wrong features are noticed at the beginning, the wrong features may be amplified in the subsequent learning process. The multi-head attention is used to enable the network to learn various different attentions, discard the wrong features and correct them in time, and extract the correct features.

[0051] As shown in Figure 5 In order to not distort the original data, the sample data calibrated in step 4 is copied and then input into three SENets for attention calculation, and three different branches are formed by using different attention transformation methods: The gating mechanism of the SENet is two fully connected layers, which generate channel vectors and the formula for generating the activation factor is as follows:

[0052]

[0053] s se =F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)

[0054] δ represents the ReLU function, r is an initial self-set hyperparameter of the SENet, which is the parameter of the two fully connected layers, F se () represents the average pooling operation, F ex () is an activation operation for generating an activation factor, s se is the generated activation factor.

[0055] Finally, the information from multiple branches is fused, and the self-adaptively extracted channel depth features are obtained by summation calculation

[0056] Step 6, contrastive representation learning

[0057] AsFigure 6 As shown, the two samples in the k-th sample pair obtained in step 2 are and After processing steps 3 to 4 in sequence, the feature vectors of the two samples in the mapping space are obtained respectively. and

[0058]

[0059]

[0060] Among them, F t represents the feature extraction operation from steps 3 to 5, and f is the convolution operation function. The fully connected layer is then used to expand, activate, and reduce the dimension of the feature vector to obtain the corresponding estimated values ​​xz1 and xz2:

[0061]

[0062]

[0063] δ' represents the ELU function, are the parameters of the two fully connected layers. By projecting, we get a mapping of a sample pair on the same low-dimensional projection space and calculate the cosine similarity z between the two. sim To return the loss function loss:

[0064]

[0065]

[0066] Where M is the set of negative sample pairs, N is the set of positive sample pairs, and bb is a bias value, which is a hyperparameter. Here, the value of the negative sample pair is divided by the value of the positive sample pair in a batch as a loss return, which can make the gap between classes larger and the distance within classes smaller.

[0067] Step 7: Classifier learning

[0068] like Figure 7 As shown, after the representation learning in step 6, the parameters of the previous representation learning stage are fixed, and the classifier is trained specifically. First, the vector O output by the representation is obtained. k , use a fully connected layer and Softmax activation function for classification, and compare the output information with the category label. Here, the fully connected layer is used as a classifier for learning, and its loss function uses a cross entropy loss (Cross Entropy Loss):

[0069]

[0070] p(x i ) represents the probability that the sample x i belongs to the positive sample, and q(x i ) represents the probability that the sample belongs to the negative sample. Using the cross-entropy loss, the mechanism of inter-class competition is adopted, the information between classes can be learned, and the cross-entropy loss only relates to the correctness of the prediction probability for the correct label, and ignores the difference of other non-correct labels, which will make the learned features more scattered. The present application uses cross-entropy loss for classifier learning and inter-class measurement loss for feature extractor, effectively combines the advantages of the two, and reduces the respective shortcomings of them to the loss of the network classification model.

[0071] The following table is the result of target appearance and disappearance detection using the existing model and the model proposed in the present method respectively:

[0072]

[0073] The present embodiment selects Acc, F1 score and BA as three indicators to measure the experimental results, wherein Acc represents the proportion of the number of samples classified correctly by the algorithm in the total number of test samples:

[0074] Acc=(TP+TN) / (TP+TN+FP+FN)

[0075] TP represents the number of positive classes predicted as positive classes, FN represents the number of positive classes predicted as negative classes, FP represents the number of negative classes predicted as positive classes, and TN represents the number of negative classes predicted as negative classes. Since the disadvantage of the evaluation index Acc is that when the classes in the sample do not match, it cannot objectively evaluate the advantages and disadvantages of the algorithm. Therefore, F1 score and BA are introduced.

[0076] F1 score=(2*Precision*Recall) / (Precision+Recall)

[0077] Precision is the proportion of true positive samples in the samples predicted as positive, Precision=TP / (TP+FP). Recall is the proportion of correctly predicted positive samples in the positive samples, Recall=TP / (TP+FN). Therefore, F1 score can take into account both precision and recall.

[0078] BA=(TP / (TP+FP)+TN / (TN+FN)) / 2

[0079] BA is used to characterize the average value of the accuracy rate between each class, in the state of sample imbalance, BA score can be used to distinguish the classification performance, and effectively obtain the local accuracy rate in each class. The higher the value, the higher the accuracy rate between each class.

[0080] Therefore, the three evaluation indicators selected in this embodiment can more objectively and comprehensively reflect the performance of the detection model. According to the table data, the method has better performance under the three evaluation scales.

Claims

1. An attention mechanism-based contrastive representation brain-machine video target detection method, characterized in that: Specifically comprising the following steps: Step 1, obtaining spatially positioned electroencephalogram data Collect the brain electrical data of the subjects when watching the video content, and segment into N samples C represents the number of electrodes, and T represents the length of the sample; whether the target appears or not is taken as the class label of the sample, wherein the target is regarded as a positive sample, and the non-target is regarded as a negative sample; Step 2, positive and negative sample pair matching Two samples of the same class are selected from the positive and negative samples respectively to form a positive sample pair, and two positive and negative samples of different classes are selected to form a negative sample pair, and the repeated sample pairs are removed; Step 3, spatial attention adaptive calibration based on electroencephalogram channel The ECAnet is used to perform squeezing and extraction operations on each electroencephalogram electrode channel of the sample, capture the interdependence and interaction between the electrodes, and generate a weight factor to represent the weight of each channel electrode; finally, the weight factor is multiplied by the electroencephalogram electrode channel of the input sample to calibrate the channel attention; Step 4, time channel attention adaptive calibration For the sample after channel attention calibration, the ECAnet is used to calculate the time channel in the sample, and a weight factor is generated to represent the weight of each time point according to the interdependence between different time dimensions, and the time attention is calibrated; Step 5, multi-head attention mechanism based channel feature depth adaptive extraction First, the sample data calibrated in step 4 is split into 3 equal branches, then input into 3 SE Nets respectively to find different spatial feature directions and generate different weights; finally, the output results of the 3 SE Nets are fused by addition to extract deep channel features; Step 6, contrastive representation learning Two samples in a sample pair obtained in step 2 are processed in turn through steps 3-5 to obtain feature vectors of the two samples in the mapping space, and then two fully connected layers are used to expand the feature vectors to calculate the cosine similarity or Euclidean distance as the distance between the two feature vectors in the sample mapping space; the distance between the positive and negative sample pairs is used to return the loss, and then back propagation is performed to make the distance between the two same samples as compact as possible and the distance between the inter-class samples as large as possible; Step 7, target detection After the back propagation training in step 6 is completed, the network parameters are fixed, the predicted class output by the classifier is compared with the true class of the sample, and the classifier is trained; then a sample with unknown class is input to determine whether there is a target and the time of target disappearance.

2. The method for target detection in brain-computer video based on comparative representation of attention mechanism as claimed in claim 1, characterized in that: The time point and position of the target appearing in the video content are random, but the appearing time length is the same.

3. The method of claim 1, wherein the method comprises: For the collected electroencephalogram data, first perform Butterworth band-pass filtering to filter to 0.1-40hz, then downsample the 1000hz electroencephalogram data to 250hz, and finally divide the continuous electroencephalogram data into multiple segments, and use the 300ms data before video playback to calibrate the data segment from 0-1000ms after video playback; then, for electroencephalogram data without target appearance, extract slices at fixed time intervals as negative samples; The electroencephalogram data with target appearance is sliced at 1s intervals, and the time periods before and after the target appearance are distinguished, and the part with target appearance is taken as the positive sample.

4. The method of claim 1, wherein the method is based on an attention mechanism. The construction method of negative sample pairs is: given the number of construction pairs And threshold T, each time two initial values ​​I1 and I2 are selected, and a sliding window pairing is performed starting from the position of I1 and I2 in the positive sample and negative sample respectively. After the number of sample pairs reaches the threshold T, I1 and I2 are randomly selected again until the number of negative sample pairs reaches 5. The method of claim 1, wherein the method is based on an attention mechanism. In step 3, first, the sample Q i Transformed from [m, n, c, t] to [m, c, n, t], where m represents the number of samples in a batch, n represents the number of feature channels, c represents the number of electrode channels, and t represents the time node; each transformed sample is input into the ECANet for squeezing and extraction operation; in step 4, the sample is transformed from [m, n, c, t] to [m, t, n, c] and then input into the ECANet.

6. The attention mechanism-based contrastive representation brain-machine video object detection method according to claim 1 or 5, characterized in that: The squeezing operation is a global average pooling operation to generate a channel descriptor: where F sq () denotes a global average pooling operation, z ic denotes the descriptor of the c-th channel in the i-th sample, Q ic denotes the signal of the c-th channel in the i-th sample q ic (j) denotes the j-th time point of the c-th channel in the i-th sample; The extraction operation is to compress the global time information of the sample into a set of channel descriptors z ic A 1x1 convolution layer with a kernel size of kernel is input, and the global time information of the sample is compressed into a set of channel descriptors z c ; wherein kernel is calculated by the following adaptive function: Wherein, abs() represents calculating absolute value, y and b are numerical fixed hyperparameters, y = 2, b = 1 in the embodiment; Through another sigmoid activation function, a dimension-increasing layer, and back to the same number of channels as the input, the channel descriptor z c is mapped into a set of channel weights, as self-attention weights on the electrode channels, to generate the activation factor s: s = F ex (z i , W) = σ(g(z i , W)) F ex () represents a full connection operation, sigma() represents a sigmoid activation function, g() represents a pooling layer and a one-dimensional adaptive convolution function, W is a parameter of a one-dimensional convolution layer, z i represents the descriptor corresponding to the i-th sample, and the last activation factor s is multiplied by the input sample brain electrical electrode channel to calibrate the channel attention. The calibrated sample data is wherein F scale (q ic ,s ic ) refers to the weight s ic and the characteristic value q ic perform a channel-based element multiplication operation.

7. The method of claim 1, wherein the method is based on an attention mechanism. The loss function used in step 6 is: where M is the set of negative sample pairs, N is the set of positive sample pairs, b is a bias value, and z is a hyperparameter; z sim denotes the cosine similarity: Z1 and z2 respectively represent the mapping of two samples in a sample pair in the same low latitude projection space.

8. The method of claim 1, wherein the method is based on an attention mechanism. Cross-entropy loss is used as the loss function H in step 7 to complete the classifier training: where p(x i ) denotes the probability that a sample x i belongs to the positive class, and q(x i ) denotes the probability that a sample belongs to the negative class.

Images