RSVP-ERP decoding method, device and medium

The spatiotemporal characteristics of EEG signals are obtained through bandpass filters and spatiotemporal attention modules, combined with PCA dimensionality reduction and task-related component analysis, the problem of insufficient identification accuracy and calculation time in the existing technology is solved, and the weak in complex background images is achieved Efficient identification of hidden targets.

CN118312771BActive Publication Date: 2024-11-12SHANGHAI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410454544.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-16
Publication Date
2024-11-12
Estimated Expiration
2044-04-16

AI Technical Summary

Technical Problem

The existing RSVP-ERP decoding method based on multi-band decomposition has insufficient recognition accuracy and calculation time when processing images of complex backgrounds under extreme conditions, especially weak and similar target and non-objective recognition, and the frequency domain-space domain Features have not received effective attention.

Method used

The bandpass filter is used to decompose the frequency band of the EEG signal, and the spatial and temporal attention module is used to obtain the spatial and temporal enhancement characteristics. The time-frequency-space filter is obtained through task-related components analysis, and the template is obtained using PCA dimensionality reduction and reorganization. Finally, it is done through two Dimensional correlation calculation is used for identification.

Benefits of technology

It improves the recognition accuracy and speed of RSVP-ERP, reduces calculation time, enhances the recognition ability of weak hidden targets, reduces noise interference, and improves the robustness of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118312771B_ABST
    Figure CN118312771B_ABST
Patent Text Reader

Abstract

The present invention relates to a RSVP-ERP decoding method, device and medium, wherein the method comprises the following steps: bandpass filtering: obtaining multi-band components of training and test data; data enhancement: enhancing each band component of training and test data; template generation: obtaining each enhanced band template of training and test data by task-related component analysis; template dimension reduction: reducing the dimension of each band template of training and test data by principal component analysis and integrating them to obtain training and test data templates; using target data template and non-target data template in training data to calculate two-dimensional correlation coefficient with test data template respectively, and judging test data category. Compared with the prior art, the present invention has the advantages of multi-domain feature capability of obtaining joint distribution of data, reducing redundancy, fast calculation speed, strong robustness, etc.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram (EEG) recognition based on event-related potential (ERP), and in particular to a RSVP-ERP decoding method, device and medium integrating a time-frequency-space attention mechanism with principal component analysis (PCA) dimensionality reduction. Background Art

[0002] With the development of artificial intelligence technology, target detection technology based on computer vision has been widely used in high-precision and rapid response fields such as real-time monitoring of police systems, environmental identification of unmanned autonomous driving, high-altitude reconnaissance of drones, and intelligence analysis of satellite photography due to its advantages in target recognition accuracy and speed. However, for images with complex backgrounds under some extreme conditions, especially weak and similar targets and non-targets (i.e., weak hidden targets and non-targets), it is still a shortcoming of computer vision processing. Therefore, the cortical coupling brain-computer interface that combines computers and human brains came into being. This brain-computer interface uses the efficient computing and data analysis capabilities of computers to process a large number of easy-to-distinguish targets with less background noise, and uses the high-precision analysis and understanding capabilities of the human brain to process weak hidden targets, forming a complementary advantage between computers and human brains, thereby achieving full coverage of the target detection field. Among them, evoked EEG and decoding technology based on the Rapid Serial Visual Presentation (RSVP) paradigm is the most commonly used technology to achieve efficient target search. At present, the method of multi-band decomposition combined with task related components analysis (TRCA) is an effective method for decoding RSVP-ERP. However, this method mainly focuses on the frequency-space domain features, and does not effectively focus on the time-frequency-space domain joint features. At the same time, multi-band components will greatly increase the redundancy of the feature space, resulting in increased algorithm calculation time and reduced recognition accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide an RSVP-ERP decoding method, device and medium that integrates time-frequency-space attention mechanism and principal component analysis dimensionality reduction. First, the electroencephalogram signal is decomposed into different frequency bands through a bandpass filter, and then the time-frequency-space enhancement characteristics of each frequency band component are obtained through a time-frequency-space attention module. Then, the time-frequency-space filter of different frequency band components is obtained through task-related component analysis, and then the templates of target and non-target electroencephalogram components in different frequency band components are respectively obtained through an integrated time-frequency-space filter group. Then, the principal components of the templates of different frequency bands are obtained through PCA dimensionality reduction and reorganized. Finally, two-dimensional correlation calculation is performed to identify the test sample, and the task-related dimensionality reduction components jointly distributed in the time-frequency-space domain are fully utilized to effectively improve the recognition accuracy and speed of RSVP-ERP.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] A RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction comprises the following steps:

[0006] Step 1) Bandpass filtering

[0007] Using N f Bandpass filter For EEG training data X σ Perform frequency band decomposition and obtain X σ N f Frequency band components Using the bandpass filter Perform frequency band decomposition on the test data Y and obtain N f Frequency band components Among them, N f ≥2, represents the preprocessed EEG training data sample set, when σ=σ 1 When represents the target EEG data sample set after preprocessing; when σ=σ 2 When Represents the non-target EEG data sample set after preprocessing; Represents single-sample test EEG data; N c Indicates the number of channels, N t Indicates the number of sampling points, N s Indicates the number of samples;

[0008] Step 2) Data Augmentation

[0009] For the frequency band components and The spatial dimension enhancement, temporal dimension enhancement and integration are performed in sequence to obtain enhanced sample sets. and

[0010] Step 3) Template generation

[0011] Obtaining enhanced sample sets through the TRCA method N f Spatial filter And obtain a template based on the spatial filter Use spatial filters Get the enhanced sample set Template

[0012] Step 4) Template dimensionality reduction

[0013] The templates were analyzed by principal component analysis. and Perform dimensionality reduction and integration to obtain training data templates Temp σ and test data template Temp (Y,σ) ;

[0014] Step 5) Template matching

[0015] For the training data template Temp σ With the test data template Temp (Y,σ) Calculate the two-dimensional correlation coefficient Based on the two-dimensional correlation coefficient, it is determined whether the test data belongs to the target EEG data, and the identification of the test data is completed.

[0016] The step 1) is specifically as follows:

[0017] Let F f The bandpass filter The fth filter in f The numerator coefficient is B f , the denominator coefficient is A f ;set up Indicates the The fth frequency band component in Indicates the The fth frequency band component in (σ,f) and Band (Y,f) The frequency band decomposition of a single channel for a single sample in is calculated as:

[0018]

[0019] In the formula, Indicates the Band (σ,f) The nth element in the cth channel of the sth sample; Represents the EEG data Xσ The nk-th element in the c-th channel of the s-th sample; Indicates the Band (σ,f) The nlth element in the cth channel of the sth sample; Indicates the Band (Y,f) The nth element in the cth channel of Y c (nk) represents the nkth element in the cth channel of the EEG data Y; Indicates the Band (Y,f) The nlth element in the cth channel of a is the feedback filter order, n b is the feedforward filter order, is the numerator coefficient B f The kth element of is the denominator coefficient A f The lth element of

[0020] The bandpass filters are used in turn. For the X σ And each channel of each sample in Y is decomposed by frequency band, and finally the and

[0021] The step 2) comprises the following steps:

[0022] Step 21) Spatial Dimension Enhancement

[0023] set up Indicates the frequency band component Band (σ,f) The sth sample enhanced in the spatial dimension, Indicates the frequency band component Band (Y,f) For samples enhanced in the spatial dimension, the calculation formula is:

[0024]

[0025] Where, Band (s,σ,f) T Indicates the Band (σ,f) The transpose of the sth sample of Indicates the Band (s,σ,f) From sampling point t+1 to N t The data segment, Indicates the Band (s,σ,f) Band is the data segment from sampling point 1 to t in (Y,f) T Indicates the Band (Y,f) The transpose of Indicates the Band (Y,f) From sampling point t+1 to N t The data segment, Indicates the Band (Y,f) The data segment from sampling point 1 to t in .

[0026] Step 22) Time dimension enhancement

[0027] set up Indicates the frequency band component Band (σ,f) The average of all samples in Project to Obtain enhanced samples in the time dimension in the spanned subspace The calculation formula is:

[0028]

[0029] In the formula, Q (σ,f) and R (σ,f) They are respectively The normal orthogonal matrix and upper triangular matrix in the QR decomposition calculation, P (σ,f) Indicates the The orthographic projection matrix of

[0030] Said Enhanced samples in the time dimension The calculation formula is:

[0031]

[0032] Step 23) Integration

[0033] Integrated enhanced single sample and The calculation formula is:

[0034]

[0035] Step 24) performs steps 21) to 23) for each frequency band component to obtain an integrated enhanced sample set on all frequency bands. and

[0036] The method for solving the spatial filter in step 3) is:

[0037] set up For the enhanced sample set The fth frequency band subset, then The s 1 and 2 The sum of the covariances between samples And the variance Var(Temp (σ,f) ) The calculation formula for the convergent constraint solution is:

[0038]

[0039] In the formula, s 1 、s 2 Indicates greater than or equal to 1 and less than or equal to N s A positive integer, c 1 、c 2 Indicates greater than or equal to 1 and less than or equal to 2N c A positive integer, Respectively indicate the The s 1 、s 2 samples; Respectively indicate the Middle 1 The sample c 1 The nth element of the channel component, the sth 2 The sample c 2 The nth element of the channel component; n represents the number of sampling points 2N t The nth point in ; Respectively The cth of all samples in 1 The nth element of the channel component, the cth 2 The nth element of the channel component; Indicates the The fth spatial filter W in (σ,f) A solution to for The transposed matrix of express c 1 、c 2 sub-element; S (σ,f) is the covariance matrix between samples of the same class and the same frequency band; V (σ,f) is the covariance matrix between sample leads of the same class and frequency band components; then the fth spatial filter W (σ,f) The optimal solution is expressed as:

[0040]

[0041] W (σ,f) That is V (σ,f) -1 S (σ,f) The eigenvalues ​​of are arranged in descending order, and the eigenvectors corresponding to them are used to find the spatial filter.

[0042] Step 3) is specifically:

[0043] set up Indicates the The template is obtained by using the spatial filter W (σ,f) For the enhanced sample set Perform spatial filtering to solve the template The calculation formula is:

[0044]

[0045] In the formula, Indicates the The sth sample in express The corresponding spatial filter The transpose of express The corresponding spatial filter The transpose of Temp (σ,f,1) Indicates the All samples in The average template after filtering, Temp (σ,f,2) Indicates the All samples in The average template after filtering, based on which the template is obtained

[0046] set up Indicates the The template is calculated as follows:

[0047]

[0048] Where Temp (Y,σ,f,1) Indicates the After the The filtered template, Temp (Y,σ,f,2) Indicates the After the The filtered template, express The corresponding spatial filter The transpose of express The corresponding spatial filter The transpose of , based on which the template is obtained

[0049] The step 4) is specifically as follows:

[0050] Template Calculate the integrated template on the fth frequency band after dimensionality reduction

[0051]

[0052] In the formula, Indicates the Temp (σ,f,1) The first p (σ,f,1) A template composed of columns, Indicates the Temp (σ,f,2) The first p (σ,f,2) A template composed of columns;

[0053] The integrated template Temp in all frequency bands σ It is expressed as:

[0054]

[0055] Template Calculate the integrated template on the fth frequency band after dimensionality reduction

[0056]

[0057] In the formula, Indicates the Temp (Y,σ,f,1) The first p (σ,f,1) A template composed of columns, Indicates the Temp (Y,σ,f,2) The first p (σ,f,2) A template composed of columns;

[0058] The integrated template Temp in all frequency bands (Y,σ) It is expressed as:

[0059]

[0060] The training data template includes a target data template and a non-target data template. In step 5), the two-dimensional correlation coefficients of the target data template and the test data template are calculated respectively. And the two-dimensional correlation coefficient between the non-target data template and the test data template if Greater than The test EEG data is determined to belong to the target EEG data, otherwise, the test EEG data is determined to belong to the non-target EEG data.

[0061] The two-dimensional correlation coefficient in step 5) The calculation process is:

[0062]

[0063] In the formula, The training data template Temp σThe element in the i-th row and j-th column of The training data template Temp σ The average of all elements of The test data template Temp (Y,σ) The element in the i-th row and j-th column of The test data template Temp (Y,σ) The average of all elements in 1 When , the two-dimensional correlation coefficient between the target data template and the test data template is When σ=σ 2 When , the two-dimensional correlation coefficient between the non-target data template and the test data template is

[0064] A RSVP-ERP decoding device integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction comprises a memory, a processor, and a program stored in the memory. When the processor executes the program, the method described above is implemented.

[0065] A storage medium stores a program, which implements the method described above when executed.

[0066] Compared with the prior art, the present invention has the following beneficial effects:

[0067] (1) The present invention adopts a multi-band decomposition method to obtain more extensive detailed features of RSVP-ERP in the frequency domain, and can efficiently identify ERP EEG induced by both conventional targets and weak hidden targets.

[0068] (2) The present invention introduces the time-frequency-space attention mechanism to obtain the multi-domain features of the joint distribution, fully mine the potential feature information, and further improve the algorithm's decoding ability for RSVP-ERP.

[0069] (3) The present invention uses PCA dimensionality reduction technology to screen a large number of redundant features, effectively avoiding the interference of noise information, reducing redundancy and improving the calculation speed of the algorithm.

[0070] (4) The present invention is based on the TRCA technology of large template splicing, which simultaneously extracts the target and non-target components in the sample and performs multi-band splicing, effectively reducing the fluctuation of the recognition results caused by the specificity of individual components, which is conducive to improving the robustness of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a flow chart of the method of the present invention;

[0072] Figure 2 It is a block diagram of the implementation principle of the present invention;

[0073] Figure 3 This is a block diagram of the data enhancement principle of the present invention;

[0074] Figure 4 Generating a flow chart for the template of the present invention;

[0075] Figure 5 This is a flow chart of template dimensionality reduction of the present invention. DETAILED DESCRIPTION

[0076] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0077] This embodiment provides a RSVP-ERP decoding method that integrates the time-frequency-space attention mechanism and PCA dimensionality reduction. The method flow is as follows: Figure 1 As shown, the principle block diagram is as follows Figure 2 As shown, the following steps are included:

[0078] Step 1) Bandpass filtering

[0079] Using N f Bandpass filter For EEG data X σ Perform frequency band decomposition and obtain X σ N f Frequency band components Using the bandpass filter Perform frequency band decomposition on the test sample Y to obtain N f Frequency band components Among them, N f ≥2; represents the preprocessed EEG training data sample set, when σ=σ 1 When represents the target EEG data sample set after preprocessing; when σ=σ 2 When Represents the non-target EEG data sample set after preprocessing; Represents the single-sample test EEG data. N c Indicates the number of channels, N t Indicates the number of sampling points and N s Indicates the number of samples.

[0080] Specifically, let F f The bandpass filter The fth filter in f The numerator coefficient is B f and the denominator coefficient is A f ;set up Indicates the The fth frequency band component in Indicates the The fth frequency band component in N c Indicates the number of channels, N t Indicates the number of sampling points and N s represents the number of samples; then the Band (σ,f) and Band (Y,f) The frequency band decomposition of a single channel for a single sample in is calculated as:

[0081]

[0082] In the formula, Indicates the Band (σ,f) The nth element in the cth channel of the sth sample; Represents the EEG data X σ The nk-th element in the c-th channel of the s-th sample; Indicates the Band (σ,f) The nlth element in the cth channel of the sth sample; Indicates the Band (Y,f) The nth element in the cth channel of Y c (nk) represents the nkth element in the cth channel of the EEG data Y; Indicates the Band (Y,f) The nlth element in the cth channel of a is the feedback filter order, n b is the feedforward filter order, is the numerator coefficient B f The kth element of is the denominator coefficient A f The lth element of .

[0083] Use the bandpass filter For the X σ And each channel of each sample in Y is decomposed by frequency band, and finally the and

[0084] Step 2) Data Augmentation

[0085] like Figure 3 As shown, First, perform spatial dimension enhancement, then perform temporal dimension enhancement, and finally perform integration to obtain the sample set: Similarly, for the Perform the same enhancement process, that is, first perform spatial dimension enhancement, then perform temporal dimension enhancement, and finally integrate to obtain the sample set Specifically, the following steps are included:

[0086] Step 21) Spatial Dimension Enhancement

[0087] set up Indicates the Band (σ,f) The sth sample enhanced in the spatial dimension, Indicates the Band (Y,f) For samples enhanced in the spatial dimension, the calculation formula is:

[0088]

[0089] Where, Band (s,σ,f) T Indicates the Band (σ,f) The transpose of the sth sample of Indicates the Band (s,σ,f) From sampling point t+1 to N t The data segment, Indicates the Band (s,σ,f) Band is the data segment from sampling point 1 to t in (Y,f) T Indicates the Band (Y,f) The transpose of Indicates the Band (Y,f) From sampling point t+1 to N t The data segment, Indicates the Band (Y,f) The data segment from sampling point 1 to t in .

[0090] Step 22) Time dimension enhancement

[0091] set up Indicates the Band (σ,f) The average of all samples in Project to Obtain enhanced samples in the time dimension in the spanned subspace The calculation formula is:

[0092]

[0093] In the formula, Q (σ,f) and R (σ,f) They are respectively The normal orthogonal matrix and upper triangular matrix in the QR decomposition calculation, P (σ,f) Indicates the The orthographic projection matrix.

[0094] Therefore, the Enhanced samples in the time dimension The calculation formula is:

[0095]

[0096] Step 23) Integration

[0097] Integrated Enhanced Single Sample and The calculation formula is:

[0098]

[0099] By analogy, the integrated enhanced sample set in all frequency bands in step 2) can be obtained: and

[0100] Step 3) Template generation

[0101] Acquired by TRCA method N f Spatial filter And get the template Then, using the spatial filter Get the Template Specifically, Figure 4 As shown, the following steps are included:

[0102] set up (N c Indicates the number of channels, N t Indicates the number of sampling points and N s represents the number of samples) is the The fth frequency band subset, then The s 1 and 2 The sum of the covariances between samples And the variance Var(Temp (σ,f) ) The calculation formula for the convergent constraint solution is:

[0103]

[0104] In the formula, s 1 、s 2 Indicates greater than or equal to 1 and less than or equal to N s A positive integer, c 1 、c 2 Indicates greater than or equal to 1 and less than or equal to 2N c A positive integer, Respectively indicate the The s 1 、s 2 samples; Respectively indicate the Middle 1 The sample c 1 The nth element of the channel component, the sth 2 The sample c 2 The nth element of the channel component; n represents the number of sampling points 2N t The nth point in ; Respectively The cth of all samples in 1 The nth element of the channel component, the cth 2 The nth element of the channel component; Indicates the The fth spatial filter W in (σ,f) A solution to for The transposed matrix of express c 1 、c 2 sub-element; S (σ,f) is the covariance matrix between samples of the same class and the same frequency band; V (σ,f) is the covariance matrix between sample leads of the same class and frequency band components; then the fth spatial filter W (σ,f) The optimal solution can be expressed as:

[0105]

[0106] W (σ,f) That is V (σ,f) -1 S (σ,f) The eigenvalues ​​of are arranged in descending order, and the spatial filter described in step 3) can be obtained by analogy.

[0107] set up Indicates the The template can be used by using W (σ,f) Regarding the Perform spatial filtering and then solve it. The calculation formula is:

[0108]

[0109] In the formula, Indicates the The sth sample in express The corresponding spatial filter The transpose of express The corresponding spatial filter The transpose of Temp (σ,f,1) Indicates the All samples in The average template after filtering, Temp (σ,f,2) Indicates the All samples in The average template after filtering. Similarly, the template described in step (3) can be obtained

[0110] Similarly, suppose Indicates the The template is calculated as follows:

[0111]

[0112] Where Temp (Y,σ,f,1) Indicates the After the The filtered template, Temp (Y,σ,f,2) Indicates the After the The filtered template, express The corresponding spatial filter The transpose of express The corresponding spatial filter The transpose of . Similarly, the template described in step 3) can be obtained

[0113] Step 4) Template dimensionality reduction

[0114] The PCA method was used to and Perform dimensionality reduction and integration to obtain training data templates Temp σ and test data template Temp (Y,σ) . Specifically, Figure 5 As shown, the following steps are included:

[0115] Assume that the template Temp (σ,f,1) Through PCA analysis, we can obtain the eigenvalues ​​λ of the 2Nc covariance matrices (σ,f,1,1) ,λ (σ,f,1,2) ,…,λ (σ,f,1,2Nc) (in descending order), then the minimum value p (σ,f,1) Should meet:

[0116]

[0117] In the formula, λ (σ,f,1,c) Represents λ (σ,f,1,1) ,λ (σ,f,1,2) ,…,λ (σ,f,1,2Nc) The c-th eigenvalue of .

[0118] Similarly, for the Temp (σ,f,2) Perform the same dimensionality reduction process:

[0119] Assume that the template Temp (σ,f,2) Through PCA analysis, we can obtain the eigenvalues ​​λ of the 2Nc covariance matrices (σ,f,2,1) ,λ (σ,f,2,2) ,…,λ (σ,f,2,2Nc) (in descending order), then the minimum value p (σ,f,2) Should meet:

[0120]

[0121] In the formula, λ (σ,f,2,c) Represents λ (σ,f,2,1) ,λ (σ,f,2,2) ,...,λ (σ,f,2,2Nc) The c-th eigenvalue of .

[0122] Then, the integrated template on the f-th frequency band after dimension reduction is It can be expressed as:

[0123]

[0124] In the formula, Indicates the Temp (σ,f,1) The first p (σ,f,1) A template composed of columns, Indicates the Temp (σ,f,2) The first p (σ,f,2) The integrated template Temp in all frequency bands σ It can be expressed as:

[0125]

[0126] Similarly, for the test sample Y, the integrated template on the fth frequency band after dimensionality reduction is It can be expressed as:

[0127]

[0128] In the formula, Indicates the Temp (Y,σ,f,1) The first p (σ,f,1) A template composed of columns, Indicates the Temp (Y,σ,f,2) The first p (σ,f,2)The integrated template Temp in all frequency bands (Y,σ) It can be expressed as:

[0129]

[0130] Step 5) Template matching

[0131] For the training data template Temp σ With the test data template Temp (Y,σ) Calculate the two-dimensional correlation coefficient That is, calculate the two-dimensional correlation coefficient of the target data template and the test data template respectively And the two-dimensional correlation coefficient between the non-target data template and the test data template if Greater than The test EEG data is determined to belong to the target EEG data, otherwise, the test EEG data is determined to belong to the non-target EEG data, and the recognition of the test data is completed.

[0132] The two-dimensional correlation coefficient in step 5) The calculation process is:

[0133]

[0134] In the formula, For the Temp σ The element in the i-th row and j-th column of For the Temp σ The average of all elements of For the Temp (Y,σ) The element in the i-th row and j-th column of For the Temp (Y,σ) The average of all elements in 1 When the two-dimensional correlation coefficient between the target data template and the test data template is When σ=σ 2 When the two-dimensional correlation coefficient between the non-target data template and the test data template is

[0135] The method of this embodiment integrates multi-band analysis technology, time-frequency and space-attention mechanism, principal component analysis dimensionality reduction technology and integrated task-related analysis technology to achieve efficient decoding of RSVP-ERP. The multi-band decomposition method can obtain the detailed features of RSVP-ERP in each frequency band, and can maintain stable and efficient recognition not only for conventional targets but also for RSVP-ERP EEG induced by weak hidden targets. The introduction of the time-frequency and space-attention mechanism can fully mine the potential and interconnected feature information in the time domain, space domain and frequency domain, and obtain the multi-domain features of joint distribution to further improve the algorithm's decoding ability for RSVP-ERP. The PCA dimensionality reduction technology is used to screen a large number of redundant features obtained, effectively avoiding the interference of noise information, reducing redundancy and improving the calculation speed of the algorithm. Based on the integrated TRCA technology of large template splicing, the target and non-target components in the sample are extracted at the same time and multi-band splicing is performed, which effectively reduces the fluctuation of the recognition results caused by the specificity of individual components, which is conducive to improving the robustness of the algorithm.

[0136] Specifically, the present embodiment utilizes a multi-frequency bandpass filter to decompose the processed EEG signal into different frequency bands, thereby extracting task-related components one by one. Compared with analyzing on the whole frequency band, the method of the present embodiment ensures that the task-related components of each frequency band are effectively extracted without being interfered by other frequency components, and finally improves the recognition accuracy of RSVP-ERP; the present embodiment utilizes the unique time-frequency-space three-domain joint distribution characteristics of RSVP-ERP. Compared with using the features of only one or two domains, the method of the present embodiment greatly expands the feature dimension of the sample by enhancing the spatial dimension and the temporal dimension on the basis of the frequency domain, thus creating favorable conditions for the extraction of task-related components; the present embodiment reduces the feature dimension by PCA dimensionality reduction. Compared with using all the features for task-related component analysis, the method of the present embodiment extracts the key information of the features by PCA and reduces the interference of noise. At the same time, reducing the feature dimension can greatly improve the calculation speed of TRCA, which is The practical application of the present invention provides a guarantee; this embodiment takes into account the target and non-target aliasing components that exist simultaneously in each type of sample, calculates the target component and non-target component template in each frequency band of each type of sample one by one through TRCA, and constructs the overall template of the sample before matching to determine whether the test sample belongs to the target or non-target. Compared with using a single component, a separate frequency band component template to match first and then integrate to determine the result, the method of this embodiment converts the aliased interference information into useful information and integrates it into the template, and has strong fault tolerance and anti-interference ability while obtaining subtle features of the data; the method of this embodiment is conducive to promoting the development of RSVP-ERP decoding technology, improving the brain-computer interface system based on ERP EEG, and can be widely used in high-precision rapid response fields such as precise monitoring of police systems, environmental identification of unmanned autonomous driving, high-altitude reconnaissance of drones, and intelligence analysis taken by satellites, and has good market application prospects and social and economic benefits.

[0137] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0138] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction, characterized in that: The following steps are involved: Step 1) Bandpass filtering Using N f Bandpass filter For EEG training data X σ Perform frequency band decomposition and obtain X σ N f Frequency band components Using the bandpass filter Perform frequency band decomposition on the test data Y and obtain N f Frequency band components Among them, N f ≥2, Represents the preprocessed EEG training data sample set. When σ=σ1, that is, represents the target EEG data sample set after preprocessing; when σ=σ2, that is, Represents the non-target EEG data sample set after preprocessing; Represents single-sample test EEG data; N c Indicates the number of channels, N t Indicates the number of sampling points, N s Indicates the number of samples; Step 2) Data Augmentation For the frequency band components and The spatial dimension enhancement, temporal dimension enhancement and integration are performed in sequence to obtain the enhanced sample sets and Step 3) Template generation Obtaining enhanced sample sets through the TRCA method N f Spatial filter And obtain a template based on the spatial filter Use spatial filters Get the enhanced sample set Template Step 4) Template dimensionality reduction The templates were analyzed by principal component analysis. and Perform dimensionality reduction and integration to obtain training data templates Temp σ and test data template Temp (Y,σ) ; Step 5) Template matching For the training data template Temp σ With the test data template Temp (Y,σ) Calculate the two-dimensional correlation coefficient Based on the two-dimensional correlation coefficient, it is determined whether the test data belongs to the target EEG data, and the identification of the test data is completed; The step 2) comprises the following steps: Step 21) Spatial Dimension Enhancement set up Indicates the frequency band component Band (σ,f) The sth sample enhanced in the spatial dimension, Indicates the frequency band component Band (Y,f) For samples enhanced in the spatial dimension, the calculation formula is: Where, Band (s,σ,f) T Indicates the Band (σ,f) The transpose of the sth sample of Indicates the Band (s,σ,f) From sampling point t+1 to N t The data segment, Indicates the Band (s,σ,f) Band is the data segment from sampling point 1 to t in (Y,f) T Indicates the Band (Y,f) The transpose of Indicates the Band (Y,f) From sampling point t+1 to N t The data segment, Indicates the Band (Y,f) The data segment from sampling point 1 to t in ; Step 22) Time dimension enhancement set up Indicates the frequency band component Band (σ,f) The average of all samples in Project to Obtain enhanced samples in the time dimension in the spanned subspace The calculation formula is: In the formula, Q (σ,f) and R (σ,f) They are respectively The normal orthogonal matrix and upper triangular matrix in the QR decomposition calculation, P (σ,f) Indicates the The orthographic projection matrix of Said Enhanced samples in the time dimension The calculation formula is: Step 23) Integration Integrated enhanced single sample and The calculation formula is: Step 24) performs steps 21) to 23) for each frequency band component to obtain an integrated enhanced sample set on all frequency bands. and 2. According to claim 1, a RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction is characterized in that: The step 1) is specifically as follows: Let F f The bandpass filter The fth filter in f The numerator coefficient is B f , the denominator coefficient is A f ;set up Indicates the The fth frequency band component in Indicates the The fth frequency band component in (σ,f) and Band (Y,f) The frequency band decomposition of a single channel for a single sample in is calculated as: In the formula, Indicates the Band (σ,f) The nth element in the cth channel of the sth sample; Represents the EEG data X σ The nk-th element in the c-th channel of the s-th sample; Indicates the Band (σ,f) The nlth element in the cth channel of the sth sample; Indicates the Band (Y,f) The nth element in the cth channel of Y c (nk) represents the nkth element in the cth channel of the EEG data Y; Indicates the Band (Y,f) The nlth element in the cth channel of a is the feedback filter order, n b is the feedforward filter order, is the numerator coefficient B f The kth element of is the denominator coefficient A f The lth element of The bandpass filters are used in turn. For the X σ And each channel of each sample in Y is decomposed by frequency band, and finally the and 3. According to claim 1, a RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction is characterized in that: The method for solving the spatial filter in step 3) is: set up For the enhanced sample set The fth frequency band subset, then The sum of the covariances between the s1th and s2th samples And the variance Var(Temp (σ,f) ) The calculation formula of the convergent constraint solution is: In the formula, s1 and s2 represent values ​​greater than or equal to 1 and less than or equal to N. s positive integer, c1 and c2 are greater than or equal to 1 and less than or equal to 2N c A positive integer, Respectively indicate the The s1th and s2th samples in ; Respectively indicate the The nth element of the c1th channel component in the s1th sample, the nth element of the c2th channel component in the s2th sample; n represents the number of sampling points 2N t The nth point in ; Respectively The nth element of the c1th channel component and the nth element of the c2th channel component of all samples in ; Indicates the The fth spatial filter W in (σ,f) A solution to for The transposed matrix of express The c1th and c2th sub-elements in S (σ,f) is the covariance matrix between samples of the same class and the same frequency band; V (σ,f) is the covariance matrix between sample leads of the same class and frequency band components; then the fth spatial filter W (σ,f) The optimal solution is expressed as: W (σ,f) That is V (σ,f) -1 S (σ,f) The eigenvalues ​​of are arranged in descending order, and the eigenvectors corresponding to them are used to find the spatial filter.

4. According to claim 3, a RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction is characterized in that: Step 3) is specifically: set up Indicates the The template is obtained by using the spatial filter W (σ,f) For the enhanced sample set Perform spatial filtering to solve the template The calculation formula is: In the formula, Indicates the The sth sample in express The corresponding spatial filter The transpose of express The corresponding spatial filter The transpose of Temp (σ,f,1) Indicates the All samples in The average template after filtering, Temp (σ,f,2) Indicates the All samples in The average template after filtering, based on which the template is obtained set up Indicates the The template is calculated as follows: Where Temp (Y,σ,f,1) Indicates the After the The filtered template, Temp (Y,σ,f,2) Indicates the After the The filtered template, express The corresponding spatial filter The transpose of express The corresponding spatial filter The transpose of , based on which the template is obtained 5. According to claim 4, a RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction is characterized in that: The step 4) is specifically as follows: Template Calculate the integrated template on the fth frequency band after dimensionality reduction In the formula, Indicates the Temp (σ,f,1) The first p (σ,f,1) A template composed of columns, Indicates the Temp (σ,f,2) The first p (σ,f,2) A template composed of columns; The integrated template Temp in all frequency bands σ It is expressed as: Template Calculate the integrated template on the fth frequency band after dimensionality reduction In the formula, Indicates the Temp (Y,σ,f,1) The first p (σ,f,1) A template composed of columns, Indicates the Temp (Y,σ,f,2) The first p (σ,f,2) A template composed of columns; The integrated template Temp in all frequency bands (Y,σ) It is expressed as:

6. According to claim 1, a RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction is characterized in that: The training data template includes a target data template and a non-target data template. In step 5), the two-dimensional correlation coefficients of the target data template and the test data template are calculated respectively. And the two-dimensional correlation coefficient between the non-target data template and the test data template if Greater than The test EEG data is determined to belong to the target EEG data, otherwise, the test EEG data is determined to belong to the non-target EEG data.

7. According to claim 6, a RSVP-ERP decoding method integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction is characterized in that: The two-dimensional correlation coefficient in step 5) The calculation process is: In the formula, The training data template Temp σ The element in the i-th row and j-th column of The training data template Temp σ The average of all elements of The test data template Temp (Y,σ) The element in the i-th row and j-th column of The test data template Temp (Y,σ) The average of all elements in ; when σ=σ1, the two-dimensional correlation coefficient between the target data template and the test data template is When σ = σ2, the two-dimensional correlation coefficient between the non-target data template and the test data template is 8. An RSVP-ERP decoding device integrating time-frequency-space attention mechanism and principal component analysis dimensionality reduction, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

9. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 7 is implemented.