Picture like-dislike tendency recognition method based on brain-computer interface

Abstract

The present application relates to brain-computer interface, consciousness recognition, for putting forward a kind of new technology of identifying the preference tendency of person, more truly obtain the real feeling of test subject, and more convenient and fast, processing speed is also relatively faster, the technical scheme adopted by the present application is, picture preference aversion tendency recognition method based on brain-computer interface, adopts fast sequence visual presentation brain interface (RSVP-BCI) paradigm, first, the target picture is shown to the test subject, then the test subject is required to identify the target picture in a series of fast flashing pictures, the brain electrical signal of human brain in this process is collected, finally, the test subject is required to score part of the target pictures;According to the subjective evaluation of test subject, the pictures are classified, and the model of electroencephalogram data under different categories of pictures is established using linear discriminant analysis (LDA) algorithm;According to the established model, the electroencephalogram data of the test subject not evaluated is classified, and the emotional tendency is obtained.The present application is mainly applied to brain-computer interface occasions.

Description

Technical Field

[0001] This invention relates to brain-computer interfaces and consciousness recognition, specifically to a method for recognizing image preferences and aversions based on brain-computer interfaces. Background Technology

[0002] Brain-computer interface (BCI) technology collects electroencephalogram (EEG) signals from the cerebral cortex and transforms these abstract brain signals into recognizable signals through a series of signal processing, feature extraction, and pattern recognition methods. This allows for the identification and prediction of human consciousness and intentions, or the manipulation of other external devices. Rapid Series Visual Presentation (RSVP) is one of the experimental paradigms of BCI based on visual stimulation. It involves rapidly presenting a sequence of images to a person in the same location. RSVP is a typical brain-computer interaction paradigm that detects targets by rapidly presenting a large number of stimulating images or videos to the brain and utilizing the ERP signals generated by the EEG signals of the scalp in response to specific material of interest. RSVP can be used to process thought information through EEG signals or to combine the human brain with a computer for rapid information retrieval and filtering. This invention, based on the RSVP paradigm in BCI, collects and processes the EEG signals of subjects when rapidly showing them images with different emotional tendencies to identify their preferences.

[0003] Brain-computer interfaces (BCIs) can rapidly analyze human brain activity, combining the human brain with machines. The RSVP experimental paradigm identifies target stimuli by detecting event-related potential (ERP) signals in the brain as images are rapidly presented sequentially. ERP signals are generated after a specific stimulus event, evoked by auditory or visual stimuli. The RSVP system classifies and identifies target stimuli primarily by relying on ERP signals in the subject's electroencephalogram (EEG) waveform after the target stimulus is presented.

[0004] The most significant characteristic of ERPs compared to other EEG components is phase locking, meaning that ERPs always appear at a fixed time after the event, and the ERP waveform is approximately the same in each trial. The remaining components in the EEG can be collectively referred to as spontaneous EEG, which does not possess phase locking. ERP component naming typically combines the timing and polarity of their appearance; for example, P300 refers to a positive peak appearing approximately 300 ms after the event presentation, and N400 refers to a negative peak appearing approximately 400 ms after the event presentation. ERP components can be divided into exogenous and endogenous components. Exogenous components are generally considered to be early components generated by external stimuli, and their amplitude, latency, and other characteristics are influenced by the physical properties of the stimulus. Endogenous components are considered to be related to human perception or cognitive activities; for example, the N170 component is closely related to face processing, and P300 is usually observed in the odd-ball paradigm, induced by a low-probability target stimulus. ERPs, especially their endogenous components, can objectively reflect changes in the nervous system during different perceptual or cognitive tasks. Therefore, they have wide applications in psychology, cognitive neuroscience, medicine, and other fields, and are also a very important research tool in brain science. In RSVP-based BCI applications, the most commonly used ERP is the P300 signal. The P300 signal appears approximately 250–750 milliseconds after the target stimulus. It is stable without training, has high amplitude, and is easily detected, thus making it widely used in the field of brain-computer interfaces.

[0005] In the field of brain-computer interfaces, the commonly used algorithm for ERP recognition is Linear Discriminant Analysis (LDA). LDA was first proposed by Ra and Fisher in 1936 for binary classification problems, hence it is also known as Fisher's Discriminant Analysis. Previous research has shown that LDA is effective on the P300-Speller, achieving good performance without excessive computation or training.

[0006] Currently, RSVP-based brain-computer interface (BCI) has been experimentally proven to identify and filter objects, scenes, people, and other relevant information and events in images or videos. In image filtering, Paul Sajda et al. used the RSVP-BCI system to filter images containing people from natural images through a single round of ERP recognition. Huang et al. also applied an RSVP-based brain-computer interface system to satellite images, using subjects' EEG signals to find images with missile bases. In image segmentation, Mohedano et al. cited the RSVP-BCI system to evaluate segmented images through a single round of ERP. In clinical diagnosis, Hope et al. developed a more effective breast cancer screening method using RSVP-based BCI in 2013. After subjects learned images with and without the target lesion, the study observed differences in P300 potential amplitude, with a greater difference between target and non-target images.

[0007] Research in the field of emotion recognition will enable machines to understand human emotions, leading to the development of more advanced brain-computer interfaces. Artificial intelligence devices will become more human-like and can also be used to detect or monitor mental health, showing broad application prospects. Currently, there are many emotion recognition technologies based on facial images, speech, and neuroimaging.

[0008] Facial expression recognition technology has only gradually developed in recent decades. Due to the diversity and complexity of facial expressions, and the involvement of physiology and psychology, expression recognition is quite challenging. Therefore, compared with other biometric technologies such as fingerprint recognition, iris recognition, and face recognition, its development has been relatively slow, and its application is not yet widespread. In the field of speech recognition, current research mainly focuses on the recognition of speech content, with less research on emotion recognition features. In speech emotion recognition, most studies classify emotions based on the basic acoustic features of the voice, using pitch, rhythm, etc., as feature inputs, and then combining them with the features of the speech content to determine the emotional state at the language level. Yamagishi et al. used conventional acoustic features as inputs and performed probability distribution statistics to determine the emotional state at the language level, achieving some success. Neuroimaging-based emotion recognition refers to acquiring physiological images such as brain MRI and near-infrared brain images of people in different emotional states, and obtaining emotional and affective characteristics through feature extraction and recognition.

[0009] However, emotion recognition based on facial images, voice, and neuroimaging each has its drawbacks. For example, facial images are easily affected by lighting conditions, voice is significantly influenced by culture, gender, age, and environmental noise, and facial images and voice can be confusing in social situations, as people may conceal their emotions for various reasons. Neuroimaging methods are expensive and slow. In contrast, emotion recognition based on physiological signals has unique advantages, such as being generated by the autonomic nervous system and not subject to conscious control. Summary of the Invention

[0010] To overcome the shortcomings of existing technologies, this invention aims to propose a new technique for identifying human preferences. Traditional preference assessment mainly relies on direct questioning, which lacks subjectivity and authenticity. Brain-computer interface (BCI) assessment can more realistically capture the subject's true feelings, and the acquisition and processing of EEG signals are more convenient and faster than other physiological imaging methods. By obtaining EEG signals from individuals facing different images, their true feelings towards those images can be obtained through signal processing, allowing for a deeper exploration of their likes and dislikes, potentially playing a greater role in psychological assessments. Therefore, the technical solution adopted in this invention is a brain-computer interface-based image preference and aversion tendency recognition method. This method employs the Rapid Sequence Visual Presentation Brain-Computer Interface (RSVP-BCI) paradigm. First, target images are shown to the subject. Then, the subject is asked to identify the target image from a series of rapidly flashing images. During this process, the subject's electroencephalogram (EEG) signals are collected. Finally, the subject is asked to rate some of the target images. Based on the subject's subjective evaluation, the images are categorized. A linear discriminant analysis (LDA) algorithm is used to establish models of EEG data for different image categories. Based on the established models, the EEG data for which the subject did not provide evaluations are categorized to obtain the subject's emotional tendency towards the images.

[0011] The detailed steps are as follows:

[0012] The images were divided into image sequence I and image sequence II. Image sequence I had already been rated by the participants as liking, neutral, and disliking. Image sequence II had not been rated for liking or disliking. To require participants to identify the images in image sequence I with labeled categories, the participants' brainwave signals were collected when image sequence I flashed sequentially on the information display interface. After data preprocessing and feature extraction, the brainwave signals were used to build a model based on the different image labels. Then, the participants were required to identify image sequence II without labeled categories. The participants' brainwave signals were collected when sequence II flashed sequentially on the information display interface. These brainwave signals were then classified according to the established model. Based on the classification results, image sequence II, which had no defined liking or disliking tendencies, was qualitatively characterized.

[0013] Further detailed steps are as follows: Participants were asked to rate their liking and dislike for picture sequence I, with scores ranging from 1 to 10, where a score of 1 represents the most disliked picture, a score of 10 represents the most liked picture, and a score of 5 represents indifference towards the picture.

[0014] The participants sat comfortably in a chair about 1 meter away from the screen. Experiment 1: The process for a single round consisted of 4 stages. The first stage was the target prompting stage. The target image that the participant needed to focus on was displayed in the center of the screen for 3 seconds. After that, the screen went out for 1 second. After the screen went out, a set of images flashed on the display screen in sequence. The images included the target image. The target image and other images flashed 10 times each, with a display duration of 150ms. The order was random. The participants needed to focus on the target image and counted silently when the target image appeared. The system collected the participants' EEG signals in real time and calculated the image that the participant was focusing on after the image flashing ended. The result image was then prompted for 2 seconds. At this time, the target images were all images in image sequence I. After the experiment, the system classified and modeled the images based on the collected EEG data and the participants' subjective evaluation.

[0015] Experiment 2 followed the same procedure as Experiment 1. Both the images displayed by RSVP and the images the participants needed to identify were from image sequence II. After the experiment, the collected EEG data were used to evaluate the images in image sequence II based on the model obtained in Experiment 1. Specifically:

[0016] The entire experiment was divided into 6 groups: 1 group was an offline experiment and 5 groups were online experiments. Each group contained 10 samples of each of the four types of tasks, for a total of 80 samples. The offline experiment was conducted first, followed by the online experiment. The EEG offline data from the first group was used to build a recognition model to decode the category of each trial in the online experiment.

[0017] The experiment used a 64-channel EEG acquisition system developed by Neuroscan. 60 channels of EEG signals ranging from 0.5 to 100 Hz were acquired using silver / silver chloride alloy electrode caps. Channels CB1, CB2, HEO, and VEO were excluded. The sampling frequency was 1000 Hz, and 50 Hz power frequency interference was filtered out. The lead distribution of the electrode caps followed the international standard 10 / 20 electrode system. The reference electrode was connected to the tip of the nose, and the ground electrode was connected to the forehead. In preprocessing, a common-average reference CAR was used to spatially filter the raw data, and the signal was downsampled to 200 Hz.

[0018] 1. Data Preprocessing

[0019] (1) Remove power frequency interference: The power frequency interference in the original signal has a large amplitude. It can be removed by using a 50Hz notch filter circuit or by using independent component analysis.

[0020] (2) Removal of electrooculography interference: The Ocular Artifact Reduction function of Scan software is used to remove electrooculography interference;

[0021] (3) Removal of baseline drift and electromyographic interference: The hardware filter built into the NeuroScan data acquisition system has a filtering range of 0.05 to 200 Hz;

[0022] (4) Superimposed and averaged ERP signals: The rhythmic potential changes that spontaneously occur in the cerebral cortex without obvious stimulation are called spontaneous EEG, with an amplitude of 10-100uV. The brain potential changes caused by external stimulation are called evoked EEG, with an amplitude of 0-100uV. A single ERP signal is usually submerged in background EEG and random noise. Therefore, signals from multiple repeated EEG experiments are superimposed to obtain a more obvious ERP signal.

[0023] The actual filtering range is 0-5.5Hz;

[0024] 2. Feature extraction and classification

[0025] First, the labels 1 and 1 are organized to facilitate subsequent classification. Images in image sequence I with scores of 8, 9, and 10 are labeled as 1; images with scores of 1, 2, and 3 are labeled as 2; and images with scores of 4, 5, 6, and 7 are labeled as -1. These labels are then grouped into a column vector. To obtain feature vectors, the experimental data is downsampled to 20Hz, and EEG data from 0-800ms after the stimulus begins (16 data points) are extracted. The data from 60 leads are connected to form a row vector. Each of the 30 target characters has 6 types × 10 rounds = 60 flashing stimuli, for a total of 30 × 60 = 1800 flashing stimuli. Therefore, a data matrix is ​​formed from 1800 feature row vectors, with each row corresponding to a label. The data matrix and labels are used to construct a classifier model, dividing the EEG data into two classes based on the labels. The LDA algorithm is used to obtain the optimal projection direction ω for the two classes of EEG data, and the model is then established.

[0026] Based on the established model, the EEG data obtained in Experiment 2 were used to calculate the decision value obtained after projection through the LDA classifier: D = ω * X J , where X J The data is represented as the j-th column of EEG data. The classification result is obtained from the decision value. The classification result is compared with the subject's actual subjective evaluation to obtain the accuracy of the system.

[0027] The features and beneficial effects of this invention are:

[0028] This invention presents a novel technology for identifying a person's likes and dislikes towards images. By introducing the RSVP paradigm, images are presented to the subject rapidly and sequentially. The task of having the subject identify the target image ensures their attention to the corresponding image. Electroencephalogram (EEG) signals generated during the process are collected and analyzed to determine the subject's emotional inclination towards the image. This technology for identifying image likes and dislikes can be used to assess a subject's mental health, the work attitude of specific employees, and can be combined with other practical applications to achieve even more advancements. Attached image description:

[0029] Figure 1 A schematic diagram of the overall structure of this invention.

[0030] Figure 2 Schematic diagram of the experimental paradigm, in which: (a) cue target (b) target character flashing (c) non-target stimulus flashing (d) blank phase.

[0031] Figure 3 Experimental timing diagram. Detailed Implementation

[0032] This invention identifies a person's emotional preferences for images using a brain-computer interface paradigm based on RSVP.

[0033] 1. Experimental Design

[0034] This invention utilizes the RSVP (Related Image Processing and Visualization) experimental paradigm to design an experiment for recognizing likes and dislikes. The RSVP paradigm involves collecting electroencephalogram (EEG) signals from the human brain as different images flash rapidly. The images presented in the experiment are divided into image sequence I and image sequence II. Image sequence I contains 30 images, which have been rated by the participants as liking, neutral, or disliking. Image sequence II contains 30 images, which have not been rated for likes or dislikes. The EEG experiment is divided into two parts. Experiment 1 requires participants to identify images in image sequence I with labeled categories. EEG signals from the human brain are collected as image sequence I flashes sequentially on the information display interface. After data preprocessing and feature extraction, a model is built based on the different image labels. Experiment 2 requires participants to identify image sequence II without labeled categories. EEG signals from the human brain are collected as sequence II flashes sequentially on the information display interface. These EEG signals are then classified according to the EEG signal model from Experiment 1. Based on the classification results, image sequence II, which has no defined likes or dislikes, is qualitatively classified. A schematic diagram is shown below. Figure 1 As shown.

[0035] Before the experiment began, participants were asked to rate their liking and dislike for picture sequence I, with scores ranging from 1 to 10. A score of 1 represented the most disliked picture, a score of 10 represented the most liked picture, and a score of 5 represented indifference towards the picture.

[0036] In Experiment 1, participants sat comfortably in a chair approximately 1 meter from the screen. The procedure for a single round of the experiment was as follows: Figure 2 As shown in (b), the experiment consisted of four phases, lasting 10 seconds. The first phase was the target cues phase, where the target image to be focused on was displayed in the center of the screen for 3 seconds, followed by a 1-second screen-off. After the screen-off, a set of images flashed sequentially on the screen, including the target image. Both the target image and other images flashed 10 times each, for a total display duration of 150 ms, in a random order. The subject was required to focus on the target image and silently count to 1 as it appeared. The system collected the subject's EEG signals in real time and calculated the image the subject was currently focusing on (the result image) after the image flashing ended, providing a 2-second cue. The target images in Experiment 1 were all from Image Sequence I, as shown in the experimental timing diagram. Figure 2 As shown in (c). The entire experiment was conducted in a quiet, undisturbed environment. After the experiment, a model was built based on the collected EEG data and the subjects' subjective evaluations.

[0037] Experiment 2 followed the same procedure as Experiment 1. Both the images displayed by RSVP and the images the participants needed to identify were from image sequence II. After the experiment, the collected EEG data were used to qualitatively evaluate the images in image sequence II based on the model obtained in Experiment 1.

[0038] The entire experiment was divided into 6 groups: 1 group was an offline experiment and 5 groups were online experiments. Each group contained 10 samples of each of the four types of tasks, for a total of 80 samples. The offline experiment was conducted first, followed by the online experiment. The offline EEG data from the first group was used to build a recognition model to decode the category of each trial in the online experiment.

[0039] The experiment used a 64-channel EEG acquisition system developed by Neuroscan, acquiring 60 channels of EEG signals ranging from 0.5 to 100 Hz (excluding CB1, CB2, HEO, and VEO channels) via silver / silver chloride (Ag / AgCl) alloy electrode caps. The sampling frequency was 1000 Hz, with 50 Hz power frequency interference filtered out. The electrode cap lead distribution followed the international standard 10 / 20 electrode system. The reference electrode was connected to the tip of the nose, and the ground electrode was connected to the forehead. In preprocessing, a common average reference (CAR) was used to spatially filter the raw data, and the signal was downsampled to 200 Hz.

[0040] 2. Data Preprocessing

[0041] Because EEG signals are weak, there are various interference signals during signal acquisition. EEG preprocessing needs to take into account interference signals and artifacts in the EEG signals. The following are the steps and methods for EEG preprocessing.

[0042] (1) Removal of power frequency interference: The amplitude of power frequency interference in the original signal is large, which can be removed by using a 50 Hz notch filter circuit. Since the EEG signal and power frequency interference are independent of each other and come from different signal sources, they can also be removed by independent component analysis (ICA).

[0043] (2) Removal of Electrooculogram (EOG) Interference: EOG is a common source of interference in EEG signals, primarily affecting EEG signals in the frontal and temporal regions. The simplest method for removing EOG interference is to delete EEG data that is significantly affected by it, but this results in the loss of much valuable data. Another method is to eliminate the EOG component from the EEG signal; the Ocular Artifact Reduction function in Scan software can remove EOG interference.

[0044] (3) Removal of baseline drift and electromyographic interference: In the experiment, the acquired EEG signals are affected by the capacitance effect between the electrodes and the scalp, as well as the influence of high-frequency electromyography. Therefore, bandpass filtering is usually used to remove interference. The hardware filter built into the NeuroScan data acquisition system has a filtering range of 0.05 to 200 Hz.

[0045] (4) Superimposed and averaged ERP signals: The rhythmic potential changes that spontaneously occur in the cerebral cortex without obvious stimulation are called spontaneous EEG, with an amplitude of 10-100uV. The brain potential changes caused by external stimulation are called evoked EEG, with an amplitude of 0-100uV. A single ERP signal is usually submerged in background EEG and random noise, so it is usually necessary to superimpose signals from multiple repeated EEG experiments to obtain a more obvious ERP signal.

[0046] In addition, due to the fast letter flashing frequency in the RSVP experimental paradigm, the average superimposed EEG signals still have a relatively obvious 6Hz periodic oscillation component, and the final actual filtering range is 0-5.5Hz.

[0047] 3. Feature extraction and classification

[0048] Simple preprocessing was performed before task recognition. First, the labels were organized to facilitate subsequent classification. Images in image sequence I with scores of 8, 9, and 10 were labeled as 1; images with scores of 1, 2, and 3 were labeled as 2; and images with scores of 4, 5, 6, and 7 were labeled as -1. These labels were then grouped into a column vector. To obtain feature vectors, the experimental data was downsampled to 20Hz, and EEG data from 0-800ms after the stimulus began (16 data points) were extracted. The data from 60 leads were concatenated to form a row vector. Each of the 30 target characters had 6 types × 10 rounds = 60 flashing stimuli, for a total of 30 × 60 = 1800 flashing stimuli. Therefore, a data matrix of 1800 feature row vectors was formed, with each row corresponding to a label. The data matrix and labels were then used to construct a classifier model.

[0049] The EEG data obtained in Experiment 2 were classified using the LDA algorithm based on the established model. The classification results were compared with the subjects' actual subjective evaluations to obtain the system's accuracy.

[0050] This invention uses the LDA algorithm to identify P300 signals in EEG signals evoked by target and non-target stimuli. This is a binary classification problem involving the identification of characters and non-target characters. The principle of binary classification LDA is to project the data to obtain the optimal projection direction ω that minimizes the intra-class distance and maximizes the inter-class distance between the two classes of data samples. The optimal projection direction ω can be obtained from the mean and variance of the original samples.

[0051] This invention designs a BCI system based on the RSVP paradigm to effectively identify people's image preference tendencies. Traditional methods for evaluating image preference / aversion lack objective evaluation methods; however, preference / aversion discrimination based on EEG signals is more direct and realistic. Experimental studies show that the ERP signals in EEG signals exhibit certain differences and separability when people receive images they like, are indifferent to, or dislike, which can be identified using machine learning algorithms. Therefore, by obtaining people's EEG signals when receiving different images, we can preliminarily determine the subjects' emotional inclinations towards these images, potentially playing a greater role in future practical applications.

[0052] This invention presents a novel technology for identifying a person's likes and dislikes towards images. By introducing the RSVP paradigm, images are presented to the subject rapidly and sequentially. The task of having the subject identify the target image ensures their attention to the corresponding image. Electroencephalogram (EEG) signals generated during the process are collected and analyzed to determine the subject's emotional inclination towards the image. This technology for identifying image likes and dislikes can be used to assess a subject's mental health, the work attitude of specific employees, and can be combined with other practical applications to achieve even more advancements.

[0053] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A brain-computer interface-based picture like-dislike tendency recognition method, characterized in that, The Rapid Sequence Visual Presentation Brain-Brain Interface (RSVP-BCI) paradigm was employed. First, target images were presented to participants. Then, participants were asked to identify the target images from a series of rapidly flashing images. Electroencephalogram (EEG) signals were collected during this process. Finally, participants were asked to rate a portion of the target images. Based on the participants' subjective ratings, the images were categorized, and a Linear Discriminant Analysis (LDA) algorithm was used to build models of the EEG data for different image categories. Based on the established models, the EEG data for which participants did not provide ratings were categorized to obtain the participants' emotional tendencies towards the images. The specific steps for categorizing images based on participants' subjective ratings and building models of the EEG data for different image categories using LDA are as follows: First, organize the tags into a column vector; Secondly, the lead data are connected to form a row vector, and the data matrix is ​​composed of column vectors and row vectors; Then, a classifier model is built using the data matrix and labels, that is, the EEG data is divided into two categories according to the labels, and the LDA algorithm is used to obtain the optimal projection direction of the two categories of EEG data. Build a model; The obtained EEG data is used to calculate decision values ​​after projection through an LDA classifier based on the established model: ,in The data is represented as the j-th column of EEG data. The classification result is obtained from the decision value. The classification result is compared with the subject's actual subjective evaluation to obtain the accuracy of the system.

2. The image preference and aversion tendency recognition method based on brain-computer interface as described in claim 1, characterized in that, The detailed steps are as follows: The images are divided into image sequence I and image sequence II. Image sequence I has already been rated by the subjects as liking, neutral, and disliking. Image sequence II has not been rated for liking or disliking. To require subjects to identify the images in image sequence I with labeled categories, the brain's electroencephalogram (EEG) signals are collected when image I flashes sequentially on the information display interface. After data preprocessing and feature extraction, a model is built based on the different image labels. Then, subjects are required to identify image sequence II without labeled categories. The brain's EEG signals are collected when sequence II flashes sequentially on the information display interface. This EEG signal is classified according to the established model. Based on the classification results, image sequence II, which has no defined liking or disliking tendencies, is qualitatively characterized.

3. The image preference and aversion tendency recognition method based on brain-computer interface as described in claim 1, characterized in that, The detailed steps are as follows: Participants were asked to rate their liking and dislike for picture sequence I, with scores ranging from 1 to 10. A score of 1 represented the most disliked picture, a score of 10 represented the most liked picture, and a score of 5 represented indifference towards the picture. The participants sat comfortably in a chair about 1 meter away from the screen. Experiment 1: The process for a single round consisted of 4 stages. The first stage was the target prompting stage. The target image that the participant needed to focus on was displayed in the center of the screen for 3 seconds. After that, the screen went out for 1 second. After the screen went out, a set of images flashed on the display screen in sequence. The images included the target image. The target image and other images flashed 10 times each, with a display duration of 150ms. The order was random. The participants needed to focus on the target image and counted silently when the target image appeared. The system collected the participants' EEG signals in real time and calculated the image that the participant was focusing on after the image flashing ended. The result image was then prompted for 2 seconds. At this time, the target images were all images in image sequence I. After the experiment, the system classified and modeled the images based on the collected EEG data and the participants' subjective evaluation. Experiment 2 followed the same procedure as Experiment 1. Both the images displayed by RSVP and the images the participants needed to identify were from image sequence II. After the experiment, the collected EEG data were used to evaluate the images in image sequence II based on the model obtained in Experiment 1. Specifically: The entire experiment was divided into 6 groups: 1 group was an offline experiment and 5 groups were online experiments. Each group contained 10 samples of each of the four types of tasks, for a total of 80 samples. The offline experiment was conducted first, followed by the online experiment. The EEG offline data from the first group was used to build a recognition model to decode the category of each trial in the online experiment. The experiment used a 64-channel EEG acquisition system developed by Neuroscan. 60 channels of EEG signals ranging from 0.5 to 100 Hz were acquired through silver / silver chloride alloy electrode caps. Channels CB1, CB2, HEO, and VEO were excluded. The sampling frequency was 1000 Hz, and 50 Hz power frequency interference was filtered out. The lead distribution of the electrode caps followed the international standard 10 / 20 electrode system, with the reference electrode connected to the tip of the nose and the ground electrode connected to the forehead. In the preprocessing, the common-average reference CAR was used to perform spatial filtering on the raw data, and the signal was downsampled to 200 Hz. 1) Data preprocessing (1) Remove power frequency interference: The power frequency interference in the original signal has a large amplitude. It can be removed by using a 50Hz notch filter circuit or by using independent component analysis. (2) Removal of electrooculography interference: The Ocular Artifact Reduction function of Scan software is used to remove electrooculography interference; (3) Removal of baseline drift and electromyographic interference: The built-in hardware filter of the NeuroScan data acquisition system has a filtering range of 0.05~200Hz; (4) Superimposed average to obtain ERP signal: The rhythmic potential changes that the cerebral cortex generates spontaneously without obvious stimulation are called spontaneous EEG, with an amplitude of 10-100uV. The brain potential changes caused by external stimulation are called evoked EEG, with an amplitude of 0-100uV. The single ERP signal is usually submerged in the background EEG and random noise. Therefore, the signals from multiple repeated EEG experiments are superimposed to obtain a more obvious ERP signal. The actual filtering range is 0-5.5Hz; 2) Feature extraction and classification First, the labels 1 and 2 are organized to facilitate subsequent classification. Images in image sequence I with scores of 8, 9, and 10 are labeled as 1; images with scores of 1, 2, and 3 are labeled as 2; and images with scores of 4, 5, 6, and 7 are labeled as -1. Then, to obtain feature vectors, the experimental data is downsampled to 20Hz, and EEG data from 0-800ms after the stimulus begins (16 data points) are extracted. The 60 leads are connected to form a row vector. Each of the 30 target characters has 6 types × 10 rounds = 60 flashing stimuli, for a total of 30 × 60 = 1800 flashing stimuli. Therefore, a data matrix is ​​formed from 1800 feature row vectors. Each row's label corresponds to a specific label. A classifier model is constructed using the data matrix and labels, dividing the EEG data into two categories based on the labels. The LDA algorithm is used to obtain the optimal projection direction for the two classes of EEG data. , and build a model.

Images