A method of investigation based on visual evoked stimulation

Abstract

The application discloses a kind of research methods based on visual evoked stimulation, comprising the following steps: in visual evoked stimulation task, the brain electrical data of participant research personnel is collected;The collected brain electrical data is screened, and effective brain electrical data is obtained;Effective brain electrical data is analyzed, and the interest degree of participant research personnel to research object is obtained;Wherein, visual evoked stimulation task is: set one of the cells in research observation grid as target cell, and target cell is used to observe picture;Each picture in picture set appears in the cell of research observation grid randomly, and only one picture appears each time;Each picture in picture set needs to be displayed once in each cell of research observation grid;Wherein, picture set includes research object and non-research object.The quality and accuracy of offline research are effectively improved, the cost of research and the requirement to research personnel are reduced, and the calculation speed is faster.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electroencephalogram signal processing, and particularly relates to a research method based on visual evoked stimulation. BACKGROUND

[0002] Offline research is a commonly used research method, which is often used in commercial market research, sociology and psychology research. Generally, offline research uses questionnaires, interviews and other means to let the subjects answer a series of questions according to the requirements. This method often cannot accurately obtain the real intention of the subjects due to the limitations of oral expression, defects in survey program design, personal bias and other reasons. Sometimes, some ingenious research methods can improve the quality of offline investigation. Such methods often require researchers to have professional skills and years of experience, and the cost is high.

[0003] Therefore, how to quickly and accurately complete the research and reduce the research cost and the requirement for researchers has become a problem to be solved urgently. SUMMARY

[0004] The application aims at the problems in the prior art, and provides a research method based on visual evoked stimulation, which converts the object to be researched into a visual picture, performs visual evoked stimulation on the research personnel in a specified position, collects electroencephalogram data, and analyzes the electroencephalogram data, thereby effectively improving the quality and accuracy of the research results.

[0005] The technical scheme is as follows:

[0006] In the visual evoked stimulation task, the electroencephalogram data of the research personnel is collected.

[0007] The collected electroencephalogram data is screened to obtain effective electroencephalogram data.

[0008] The effective electroencephalogram data is analyzed to obtain the interest degree of the research personnel to the research object.

[0009] The visual evoked stimulation task is as follows:

[0010] One of the cells in the research observation grid is set as a target cell, and the target cell is used for observing the picture set.

[0011] Each picture in the picture set is randomly displayed in the cell of the research observation grid, and only one picture is displayed in the cell each time. Each picture in the picture set needs to be displayed in each cell of the research observation grid once.

[0012] The picture set includes the research object and a non-research object.

[0013] The method for preprocessing the collected EEG data comprises preprocessing and low-quality data processing.

[0014] The method for preprocessing the collected EEG data comprises preprocessing and low-quality data processing. The method for preprocessing the collected EEG data comprises preprocessing and low-quality data processing.

[0015] The collected EEG data of the participants in the research is filtered. The collected EEG data of the participants in the research is filtered.

[0016] The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated. The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated.

[0017] The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated. The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated.

[0018] The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated. The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated.

[0019] The EEG data of the first second in the original EEG sample of each picture display is taken as the baseline, and each lead of the original EEG sample of each picture display is individually baseline calibrated.

[0020] Further, the visual evoked stimulation task comprises a plurality of target cells, and the target cells in each visual evoked stimulation task are different.

[0021] Further, in the visual evoked stimulation task, the eye movement data of the participants in the research is collected, and the EEG data is screened through the eye movement data to obtain effective EEG data.

[0022] Further, in the visual evoked stimulation task, the eye movement data of the participants in the research is collected, and the EEG data is screened through the eye movement data to obtain effective EEG data.

[0023] Further, in the visual evoked stimulation task, the eye movement data of the participants in the research is collected, and the EEG data is screened through the eye movement data to obtain effective EEG data.

[0024] Further, the analysis of the target electroencephalogram signal comprises: obtaining each electroencephalogram sample classification value by cross-validation on the classifier; calculating the average value of the classification value corresponding to each picture displayed in the target cell; combining the serial number of the picture, and sorting the average value of the classification value from high to low to obtain the interest degree of the research object of the research personnel.

[0025] The cross-validation is to combine the effective electroencephalogram data, select one combination as a classification combination, use the trained classifier to classify each sample in the classification combination, obtain the classification value of each sample in the classification combination; repeat the training and classification process of all combinations as the classification combination to obtain the classification value of all samples.

[0026] Further, the method for obtaining the interest degree of the research object of the research personnel further comprises: obtaining the feature information of the electroencephalogram data and the picture perception as the electroencephalogram feature value corresponding to the target picture number; and sorting the electroencephalogram feature value from high to low to obtain the interest degree of the research object of the research personnel; wherein the feature information comprises an electroencephalogram peak value feature and an electroencephalogram energy feature; the electroencephalogram feature value is the average value of the electroencephalogram peak value feature and the electroencephalogram energy feature; the electroencephalogram peak value feature is the maximum value of the electroencephalogram signal within 100-400 milliseconds of the electrode on the top of the head; and the electroencephalogram energy feature is the sum of the absolute values of the electroencephalogram within 100-400 milliseconds of the electroencephalogram sample when each picture is displayed.

[0027] The application also provides a computer readable medium storing software, the software including instructions executable by one or more computers, the instructions causing the one or more computers to perform operations when executed by the one or more computers, the operations including the flow of the above-mentioned research method based on visual evoked stimulation.

[0028] The application also provides a computer system, comprising:

[0029] one or more processors;

[0030] a memory storing instructions operable to cause the one or more processors to perform operations when executed by the one or more processors, the operations including the flow of the above-mentioned research method based on visual evoked stimulation.

[0031] Working principle: This invention transforms the subjects to be surveyed into visual images, provides visual stimuli to the survey participants at specified locations, and then collects and analyzes the EEG data to determine the survey participants' level of interest in the subjects. Simultaneously, this invention incorporates an eye tracker to filter the EEG data, eliminating low-quality data.

[0032] Beneficial effects: Compared with existing technologies, this invention effectively improves the quality and accuracy of offline surveys, while reducing survey costs and the requirements for survey personnel. Furthermore, the method provided by this invention yields results faster and is applicable to a wider range of fields. Attached Figure Description

[0033] Figure 1 This is a flowchart of the process of Example 1;

[0034] Figure 2 This is a schematic diagram of the survey observation grid used in Example 1;

[0035] Figure 3 This is a flowchart of the process of Example 3. Detailed Implementation

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1:

[0038] This embodiment provides a survey method based on visually evoked stimuli, such as... Figure 1 As shown, it includes the following steps:

[0039] Step 1: Collect EEG data from the participants in the visual evoked stimulus task.

[0040] First, participants in the survey are fitted with electroencephalogram (EEG) measurement devices. These devices typically consist of electrode caps, with electrode distribution conforming to the international 10-20 standard. The number of electrodes used depends on the survey plan, equipment cost, and survey duration. When wearing the EEG device, it is crucial to ensure the electrodes are accurately positioned and that their impedance meets requirements; calibration is necessary before use.

[0041] The visual evoked stimulus task is as follows:

[0042] 1) Identify the target cell; at this point, a survey observation grid including d1 cells is displayed on the screen, with each cell labeled. The screen will indicate the target cell number, such as... Figure 2 As shown, this embodiment uses a nine-square grid as the survey observation grid, and d1 is 9 in this embodiment;

[0043] 2) Images from the image set will randomly appear in the cells of the survey observation grid; the image set contains N1 images, including N2 survey images and N3 non-survey images, where N1 = N2 + N3.

[0044] In this process, each image is displayed once in each cell of the survey observation grid, constituting one image display cycle. A complete cycle of image display occurs when all images in the image set have been displayed once in each cell. During each cycle, only one image is displayed at a time, and it appears only in any cell of the survey observation grid. Each image in the image set is displayed only once, and the cycle ends after all images in the image set have been displayed. To ensure the effectiveness of the EEG signals, two adjacent images cannot appear in the same cell. During each cycle, each image is displayed for 300 milliseconds, and the interval between adjacent image displays is 400 milliseconds.

[0045] In order to increase the amount of data, multiple rounds of image display can be performed in each visual evoked stimulus task. In this embodiment, 11 rounds of image display are performed.

[0046] During the visually evoked stimulus task, researchers simply observe the image in the target cell based on the prompts on the screen.

[0047] Step 2: Filter the collected EEG data to obtain valid EEG data. This filtering process includes preprocessing and processing of low-quality data.

[0048] Step 20: Data preprocessing;

[0049] Step 201: Perform a 0.5-40Hz bandpass filter on the EEG data collected in Step 1;

[0050] Step 202: Extract the EEG data from the EEG data obtained in Step 201, from 1 second before the image appears to 1 second after the image appears each time the image is displayed, and obtain the EEG data for each time the image is displayed, as the original EEG sample for each time the image is displayed.

[0051] Step 203: Baseline calibration is performed on each lead of the original EEG sample for each picture presentation, respectively, taking the first second of EEG data in the original EEG sample for each picture presentation as the baseline; and EEG samples for each picture presentation are formed by extracting data from the time when the picture appears to 1 second after the picture appears in each piece of data, respectively;

[0052] Step 204: Labels for the EEG samples for each picture presentation are set, respectively; and the labels include the following information:

[0053] 1) Target information; if the presented picture appears in the target cell, the sample is marked as a target, otherwise it is marked as a non-target. The target label is 1 and the non-target is 0.

[0054] 2) Picture set picture sequence number n1; the picture sequence number is the sequence number of the picture in the picture set; the picture sequence number n1 ranges from 1 to N1; in this embodiment, the picture sequence number n1 in the picture set is constant.

[0055] 3) Group sequence number n2; each group of pictures corresponds to a group sequence number n2 in chronological order; the group sequence number n2 ranges from 1 to N4; in this embodiment, N4 = N1 x l1; l1 represents the number of rounds of picture presentation in the visual evoked stimulation task; and there are d1 EEG samples in the EEG sample set corresponding to each group of pictures.

[0056] 4) Presentation picture sequence number n3; the presented pictures are numbered in chronological order; the presentation picture sequence number n3 ranges from 1 to N5; in this embodiment, N5 = N1 x d1 x l1; l1 represents the number of rounds of picture presentation in the visual evoked stimulation task.

[0057] Step 205: Steps 202-204 are repeated to obtain EEG data samples for each picture presentation in the visual evoked stimulation task and labels for the EEG samples for each picture presentation; and a first EEG sample set is formed by all EEG samples for each picture presentation and all labels for the EEG samples for each picture presentation. The dimensions of all EEG samples for each picture presentation are N5 x number of leads x 1 second of data points. The dimensions of all labels for the EEG samples for each picture presentation are N4 x 4.

[0058] Step 21: Low-quality data processing: EEG samples with a maximum absolute value greater than a first threshold value in the EEG samples for each picture presentation in the first EEG sample set are deleted; and valid EEG data is obtained; and the process includes the following steps:

[0059] Step 211: In the first EEG sample set, EEG samples corresponding to all group sequence numbers n2 are extracted according to the group sequence number n2;

[0060] Step 212: Extract the maximum value of the absolute value of the EEG sample when each picture is displayed in all the EEG samples with group number n2 extracted in step 211; if the value is greater than the first threshold value v1, the EEG sample corresponding to the displayed picture is removed. The value of the first threshold value v1 depends on the data processing, reference, and grounding method of the EEG acquisition device. Most EEG data does not exceed 50 μV. In this embodiment, the first threshold value v1 is 100 μV.

[0061] Step 213: If the removed EEG sample corresponding to the displayed picture is the target sample, or the total amount of removed data exceeds 1 / 3 of all samples in the group, remove all samples in the group. The target sample represents the EEG sample generated when the picture appears in the target cell.

[0062] Step 214: n2+1, repeat steps 211-213 until all groups are traversed. The second EEG sample set obtained is the valid EEG data.

[0063] Step 215: If the removed EEG sample exceeds 20% of the total number of EEG samples in the first EEG sample set, it indicates that the data acquisition quality is very poor, and the subsequent operation is terminated. If not, continue to step 3.

[0064] Step 3: Analyze the valid EEG data to obtain the interest level of the participants in the research object; mainly including the following steps:

[0065] Step 30: Downsample the second EEG sample set to reduce the sampling rate to 50 Hz. The labels of the samples remain unchanged to form the third EEG sample set.

[0066] Step 31: Shuffle the third EEG sample set, divide it into N6 different combinations with the group as the minimum unit, and form the fourth EEG sample set with the combination number n4 ranging from 1 to N6.

[0067] The number of combinations can exceed or be less than 10. The more combinations, the longer the calculation time, but the higher the accuracy. Among them, the sample combinations meet the following requirements:

[0068] 1) Samples with the same group number form a sample group.

[0069] 2) A sample group is in the same combination and can only belong to one combination.

[0070] 3) Each sample group is randomly divided into different combinations.

[0071] 4) All samples in the third EEG sample set are in the fourth EEG sample set, and the number of samples is consistent.

[0072] 5) In each combination, the number of samples in the sample group is consistent or close to consistent.

[0073] Step 32: Start cross-validation, each sample gets a classification value, as the first electroencephalogram classification set. The larger the value, the more likely the sample corresponds to the picture that appears on the target cell.

[0074] The cross-validation steps are as follows:

[0075] Step 321: In the fourth electroencephalogram sample set, extract all electroencephalogram sample sets corresponding to the combination sequence n4, and retain the target information in the label of the electroencephalogram sample in the electroencephalogram sample set corresponding to the combination sequence n4 to form a first electroencephalogram temporary sample set;

[0076] Step 322: Extract the electroencephalogram sample set of all combinations except the combination sequence n4 in the fourth electroencephalogram sample set, and only retain the target information in the label of the electroencephalogram sample to form a second electroencephalogram temporary sample set;

[0077] Step 323: Use the second electroencephalogram temporary sample set to train the classifier, and the trained classifier is the first temporary classifier. The classifier can be SVM, neural network or other classifier; In the training process, the electroencephalogram sample in the second electroencephalogram temporary sample set is used as the input signal, and the corresponding target information is used as the output to train the parameters in the classifier.

[0078] Step 324: Use the first temporary classifier to classify each electroencephalogram sample in the first electroencephalogram temporary sample set. For each electroencephalogram sample, the classifier will give a classification value;

[0079] Step 325: n4+1, repeat steps 321-324 until all electroencephalogram samples in all combinations get a classification value. The classification value of each electroencephalogram sample and the label information of each electroencephalogram sample are combined to form the first electroencephalogram classification set.

[0080] Step 33: Process the first electroencephalogram classification set to obtain the first electroencephalogram result set.

[0081] A set of electroencephalogram samples should contain d1 electroencephalogram samples, representing pictures appearing on different cells. Only one picture appears on the target cell, and the others appear on the non-target cell. The subject can only observe the picture appearing on the target cell and generate the corresponding electroencephalogram signal. Correspondingly, the classification value of the picture appearing on the target cell should also be the largest in the group.

[0082] Step 331: Take a group of data from the first electroencephalogram classification set as the first electroencephalogram temporary classification set.

[0083] Step 332: Find the sample corresponding to the picture sequence number of the target information in the first electroencephalogram temporary classification set, as the target picture number, and record it.

[0084] Step 333: Find the target information corresponding to the sample with the maximum classification value in the first temporary EEG classification set as the classification result of the group, and record it. If the target information corresponding to the sample with the maximum classification value is 1, it means that the subject can successfully identify the picture of interest through the EEG data, otherwise it means that it cannot.

[0085] Step 334: Repeat steps 331-333 until all groups are calculated, and the data recorded in steps 332 and 333 form the first EEG result set.

[0086] Step 34: According to the first EEG result set, the interest degree of the participants in the research object is obtained. Specifically, the following steps are included:

[0087] Step 341: Calculate the average value of the classification result corresponding to each target picture number in the first EEG result set; this average value is the classification accuracy of the picture appearing on the target cell.

[0088] Step 342: According to the correspondence between the research object and the target picture number, the classification accuracy is sorted from high to low to obtain the interest degree of the participants in the research object. The higher the classification accuracy of the research object, the greater the interest degree of the participants.

[0089] Embodiment 2:

[0090] The embodiment provides a research method based on visual evoked stimulation, which includes the following steps:

[0091] Step 1: In the visual evoked stimulation task, collect the EEG data of the participants.

[0092] First, the participants are equipped with an EEG measurement device, which is usually an electrode cap. The electrode distribution is according to the international 10-20 standard, and the number of electrodes is determined according to the research plan, equipment cost, and research time. When wearing the EEG acquisition device, attention should be paid to whether the electrode position is accurately placed and whether the electrode impedance meets the requirements. Calibration before use.

[0093] The visual evoked stimulation task is: 1) Determine the target cell; at this time, a research observation grid including d1 cells is displayed on the screen, and the cell is labeled with a label. The target cell is indicated on the screen. In this embodiment, a nine-square grid is used as the research observation grid, and d1 is 9 in this embodiment.

[0094] 2) Randomly appear pictures in the cell in the research observation grid; the picture set includes N1 pictures, including N2 research pictures and N3 non-research pictures, N1=N2+N3.

[0095] In the grid, each cell displays a picture once, and the picture set is a picture display. All pictures in the picture set are displayed once in each cell, and each picture is displayed once in each picture display. After all pictures in the picture set are displayed, the picture display is over. In order to ensure the effect of the electroencephalogram, the two pictures appearing in succession cannot appear in the same cell. In each picture display, each picture is displayed for 300 milliseconds, and the interval between the two pictures is 400 milliseconds.

[0096] In order to increase the amount of data, multiple picture displays can be performed in each visual evoked stimulation task. In this embodiment, 11 picture displays are performed. At the same time, multiple visual evoked stimulation tasks are added in this embodiment.

[0097] In a visual evoked stimulation task, the investigator observes the picture in the target cell according to the prompt on the screen. After the visual stimulation task is over, a new visual stimulation task will start, and the investigator will be prompted to observe the new target cell on the display screen.

[0098] Step 2: The collected electroencephalogram data is screened to obtain effective electroencephalogram data. The screening of the collected electroencephalogram data includes preprocessing and low-quality data processing.

[0099] Step 20: Data preprocessing;

[0100] Step 201: The electroencephalogram data collected in step 1 is subjected to 0.5-40Hz band-pass filtering;

[0101] Step 202: Extract the electroencephalogram data from 1 second before the picture appears to 1 second after the picture appears in the electroencephalogram data obtained in step 201, to obtain the electroencephalogram data for each picture display as the original electroencephalogram sample for each picture display;

[0102] Step 203: The first second of electroencephalogram data in each original electroencephalogram sample for each picture display is taken as the baseline, and each lead of the original electroencephalogram sample for each picture display is individually baseline calibrated; the data from the appearance of the picture to 1 second after the appearance of the picture in each data segment is intercepted to form the electroencephalogram sample for each picture display;

[0103] Step 204: The electroencephalogram sample for each picture display is labeled respectively; the label contains the following information:

[0104] 1) Target information; if the displayed picture appears in the target cell, the sample is marked as target, otherwise it is marked as non-target. The target label is 1 and the non-target is 0.

[0105] 2) picture sequence number n1 in the picture set, the picture sequence number is the sequence number of the picture in the picture set, the picture sequence number n1 ranges from 1 to N1, and the picture sequence number n1 in the embodiment is constant.

[0106] 3) group sequence number n2, each group of pictures corresponds to a group sequence number n2 in chronological order, the group sequence number n2 ranges from 1 to N4, N4 = N1 x l1 x P in the embodiment; l1 represents the number of rounds of picture presentation in the visual evoked stimulation task. There are d1 electroencephalogram samples in the electroencephalogram sample set corresponding to each group of pictures, and P represents the total number of visual evoked stimulation tasks.

[0107] 4) presentation picture sequence number n3, the presented pictures are numbered in chronological order, the presentation picture sequence number n3 ranges from 1 to N5, N5 = N1 x d1 x l1 x P in the embodiment; l1 represents the number of rounds of picture presentation in the picture set in the visual evoked stimulation task, and P represents the total number of visual evoked stimulation tasks.

[0108] Step 205: repeat steps 202-204; obtain the electroencephalogram data sample of each presented picture in all visual evoked stimulation tasks and the label of the electroencephalogram sample of each presented picture; and form a first electroencephalogram sample set by combining the labels of all electroencephalogram samples of all presented pictures and all electroencephalogram samples of all presented pictures in all visual evoked stimulation tasks. The dimension of all electroencephalogram samples of all presented pictures is N5 x number of leads x number of 1-second data points. The dimension of all electroencephalogram sample labels of all presented pictures is N4 x 4.

[0109] Step 21: low-quality data processing: delete the electroencephalogram sample whose maximum absolute value of the electroencephalogram sample of each presented picture in the first electroencephalogram sample set is greater than a first threshold value; obtain valid electroencephalogram data; specifically including the following steps:

[0110] Step 211: extract the electroencephalogram sample corresponding to all group sequence numbers n2 in the first electroencephalogram sample set according to the group sequence number n2;

[0111] Step 212: extract the maximum absolute value of the electroencephalogram sample of each presented picture in the electroencephalogram sample of all group sequence numbers n2 extracted in step 211; if the value is greater than a first threshold value v1, the electroencephalogram sample corresponding to the presented picture is removed. The value of the first threshold value v1 depends on the data processing, reference, and grounding method of the electroencephalogram acquisition device, and most electroencephalogram data does not exceed 50 μV. In the embodiment, the first threshold value v1 is 100 μV.

[0112] Step 213: if the removed electroencephalogram sample corresponding to the presented picture is a target sample, or the total amount of removed data exceeds 1 / 3 of all samples in the group, all samples in the group are removed. The target sample represents the electroencephalogram sample generated when the picture appears in the target cell.

[0113] Step 214: n2+1, repeat steps 211-213 until all groups are traversed. The second set of EEG samples is obtained, which is the valid EEG data.

[0114] Step 215: If the number of removed EEG samples exceeds 20% of the total number of EEG samples in the first set of EEG samples, it indicates that the quality of this data collection is poor, and the subsequent operation is terminated. If not, continue step 3.

[0115] Step 3: Analyze the valid EEG data to obtain the interest level of the participants in the research object; mainly including the following steps:

[0116] Step 30: Downsample the second set of EEG samples to reduce the sampling rate to 50Hz. The labels of the samples remain unchanged, forming a third set of EEG samples;

[0117] Step 31: Shuffle the third set of EEG samples, and divide it into N6 different combinations with the group as the minimum unit. The combination sequence number n4 ranges from 1 to N6, forming a fourth set of EEG samples.

[0118] The number of combinations can be more or less than 10. The more combinations, the longer the calculation time, but the higher the accuracy. Among them, the sample combination meets the following requirements:

[0119] 1) Samples with the same group sequence number form a sample group.

[0120] 2) A sample group is in the same combination and can only belong to one combination.

[0121] 3) Each sample group is randomly divided into different combinations.

[0122] 4) All samples in the third set of EEG samples are in the fourth set of EEG samples, and the number of samples is consistent.

[0123] 5) In each combination, the number of samples in the sample group is consistent or close to consistent.

[0124] Step 32: Start cross-validation, and each sample obtains a classification value as the first EEG classification set. The larger the value, the more likely it is that the picture corresponding to the sample is the picture appearing on the target cell.

[0125] The cross-validation steps are as follows:

[0126] Step 321: Extract all EEG sample sets corresponding to the combination sequence number n4 from the fourth set of EEG samples, and retain the target information in the labels of the EEG samples in the EEG sample set corresponding to the combination sequence number n4. Form a first temporary set of EEG samples;

[0127] Step 322: Extract all the EEG sample sets except the combination index n4 from the fourth EEG sample set, and only keep the target information in the label of the EEG sample, forming a second EEG temporary sample set;

[0128] Step 323: Train the classifier using the second EEG temporary sample set, and the trained classifier is the first temporary classifier. The classifier can be SVM, neural network or other classifiers; in the training process, the EEG samples in the second EEG temporary sample set are used as input signals, and the corresponding target information is used as output to train the parameters in the classifier.

[0129] Step 324: Classify each sample in the first EEG temporary sample set using the first temporary classifier. For each EEG sample, the classifier will give a classification value;

[0130] Step 325: n4+1, repeat steps 321-324 until all the EEG samples in the combination get a classification value. The classification value of each EEG sample and the label information of each EEG sample are combined to form the first EEG classification set.

[0131] Step 33: Process the first EEG classification set to obtain the first EEG result set.

[0132] A set of EEG samples should contain d1 EEG samples, representing pictures appearing on different cells. Only one picture appears on the target cell, and the others appear on non-target cells. The participants can only observe the pictures appearing on the target cell and generate corresponding EEG signals. Correspondingly, the classification value of the picture appearing on the target cell should also be the largest in the group.

[0133] Step 331: Take all the samples in the first EEG classification set with the group index equal to the combination index as the first EEG temporary classification set.

[0134] Step 332: Find the picture sequence number corresponding to the sample with target information 1 in the first EEG temporary classification set as the target picture number, and record it.

[0135] Step 333: Find the target information corresponding to the sample with the largest classification value in the first EEG temporary classification set as the classification result of the group, and record it. If the target information corresponding to the sample with the largest classification value is 1, it means that the picture the subject pays attention to can be successfully identified through EEG data, otherwise it means that it cannot be identified.

[0136] Step 334: Increase the group index by 1, repeat steps 331-333 until all groups obtain the target picture number and the classification result of the group to form the first EEG result set.

[0137] Step 34: judging the interest degree of the research participants to the research objects according to the first electroencephalogram result set and the second electroencephalogram sample set. The following steps are included:

[0138] Step 341: extracting the electroencephalogram sample with the target information of 1 as the first electroencephalogram temporary sample in the second electroencephalogram sample set according to the group sequence number n2. If the group cannot be found, it means that the group has been eliminated, and the current loop is skipped.

[0139] Step 342: extracting the data with the classification result of 1 in the first electroencephalogram result set according to the group sequence number n2. If the group cannot be found, it means that the group classification is wrong, and the current loop is skipped.

[0140] Step 343: counting the feature information related to the picture perception in the electroencephalogram data as the electroencephalogram feature value corresponding to the target picture number. In the embodiment, the feature information includes the peak value feature and the energy feature, which are selected as the computer data at the top of the head, the P300 component of the electroencephalogram data, and the calculation method is as follows: counting the first electroencephalogram temporary sample, the Cz lead (or other electrodes at the top of the head), the maximum value within 100-400 milliseconds as the peak value feature. Taking the sum of the absolute values of the electroencephalogram within the first 100-400 milliseconds as the energy feature, and recording;

[0141] Step 344: repeating steps 341-343 with the group sequence number n2+1 until all group data are traversed.

[0142] Step 345: normalizing all feature values according to the type of electroencephalogram feature value. Taking the peak value feature as an example, the specific calculation steps are as follows: first, calculate the average value and standard deviation of all peak value features. Then, subtract the average value from the peak value data, and then divide by the standard deviation to get the new peak value feature. The new energy feature uses the same calculation method as the new peak value feature.

[0143] Step 346: counting the average value of the new peak value feature and the new energy feature for different target picture numbers (without distinguishing the feature categories) as the electroencephalogram feature value of the picture number.

[0144] Step 347: according to the correspondence between the research objects and the target picture numbers, the electroencephalogram feature values are sorted from high to low to obtain the interest degree order of the research participants to the research objects.

[0145] Embodiment 3:

[0146] The embodiment provides a research method based on visual evoked stimulation, as shown in Figure 3 The following steps are included:

[0147] Step 1: collecting the electroencephalogram data and eye movement data of the research participants in the visual evoked stimulation task.

[0148] First, the EEG measurement device is worn by the participants, and the eye tracker is placed in the appropriate position and calibrated. The EEG acquisition device is usually an electrode cap, and the electrode distribution follows the international 10-20 standard. The number of electrodes used is determined comprehensively according to the research plan, equipment cost, and research time. When wearing the EEG acquisition device, attention should be paid to whether the electrode position is accurately placed and whether the electrode impedance meets the requirements. The eye tracker is placed in front of the participants and should not obstruct the display of the research object. It is calibrated before use.

[0149] The visual evoked stimulation task is as follows: 1) Determine the target cell; at this time, a research observation grid including d1 cells is displayed on the screen, and a label is marked in the cell. The target cell is indicated as a number on the screen, such as Figure 1 In this embodiment, a nine-square grid is used as the research observation grid, and d1 is 9 in this embodiment.

[0150] 2) Randomly display pictures in the cells of the research observation grid from the picture set; the picture set includes N1 pictures, including N2 research pictures and N3 non-research pictures, and N1 = N2 + N3.

[0151] In each cell of the research observation grid, a set of pictures is displayed once, and all pictures in the picture set are displayed once in each cell in each round of picture display. In each round of picture display, only one picture is displayed at a time, and only in any one cell of the research observation grid. Each picture in the picture set is displayed only once, and the round of picture display ends after all pictures in the picture set are displayed. In order to ensure the effectiveness of the EEG signal, the two pictures that appear adjacent to each other cannot appear in the same cell. In each round of picture display, each picture is displayed for 300 milliseconds, and the interval between adjacent pictures is 400 milliseconds.

[0152] In order to increase the amount of data, multiple rounds of picture display can be performed in each visual evoked stimulation task. In this embodiment, l1 rounds of picture display are performed. At the same time, multiple visual evoked stimulation tasks are added in this embodiment.

[0153] During a visual evoked stimulation task, the participant observes the picture in the target cell according to the prompt on the screen. After the completion of a visual stimulation task, a new visual stimulation task is started directly, and the participant is prompted to observe the new target cell on the display screen.

[0154] Step 2: Process the collected eye movement data and EEG data, and use the eye movement data to filter the EEG data, and finally obtain valid EEG data.

[0155] The processing of eye movement data mainly includes the following steps:

[0156] Step 2-11: Organize the eye tracker data to form the first eye movement sample set. After the data of the eye tracker is organized, the gaze focus coordinate data in the time dimension is obtained, which indicates which position on the screen the participant is looking at at a certain time. The processing method of coordinate data is prior art. The number of samples per second is 100, and each sample contains a two-dimensional coordinate point.

[0157] Step 2-12: Extract eye movement coordinate data from 0 seconds after the appearance of each picture to t2 seconds after the appearance of the picture from the eye tracker data to form a coordinate sample set.

[0158] Step 2-13: Set the label of each eye movement data sample, which contains the following information:

[0159] 1) Target information; if the presented picture appears in the target cell, the sample is marked as target, otherwise it is marked as non-target. The target label is 1 and the non-target is 0.

[0160] 2) Picture position number, which is the cell number, the range is 1~d1.

[0161] 3) Group number n2, according to the time sequence, each group of pictures corresponds to a group number n2, the range of group number n2 is 1 to N4, N4=N1x11xP in this embodiment; 11 represents the number of rounds of picture presentation in the visual evoked stimulation task. Each group of pictures corresponds to d1 EEG samples in the EEG sample set, and P represents the total number of visual evoked stimulation tasks.

[0162] 4) Presentation picture order number n3, which is numbered according to the time development order, the range of presentation picture order number n3 is 1 to N5, N5=N1x d1x 11x P in this embodiment; 11 represents the number of rounds of picture presentation in the visual evoked stimulation task, and P represents the total number of visual evoked stimulation tasks.

[0163] Step 2-14: The coordinate sample and the label together form the first eye movement sample set. The dimension of the coordinate sample is N3x2xdata points. The dimension of the label is N3x3.

[0164] Wherein, the processing of the collected EEG data includes the following steps:

[0165] Step 2-21: 0.5-40Hz band-pass filtering of the collected EEG data in step 1;

[0166] Step 2-22: Extract the EEG data from 1 second before the appearance of the picture to 1 second after the appearance of the picture from the EEG data obtained in step 2-21 each time the picture is presented, to obtain the EEG data of each picture presentation as the original EEG sample of each picture presentation.

[0167] Step 2-23: Baseline calibration is performed on each electrode of the original EEG sample for each picture presentation, with the first second of EEG data in the original EEG sample as the baseline for each picture presentation; the EEG data from the picture presentation to 1 second after the picture presentation is extracted from each piece of data to form an EEG sample for each picture presentation;

[0168] Step 2-24: The EEG sample for each picture presentation is labeled respectively; the label contains the following information:

[0169] 1) Target information; if the presented picture appears in the target cell, the sample is marked as target, otherwise it is marked as non-target. The target label is 1 and the non-target is 0.

[0170] 2) Picture set picture sequence number n1, picture sequence number is the sequence number of pictures in the picture set, picture sequence number n1 ranges from 1 to N1, and in this embodiment, picture sequence number n1 in the picture set is constant.

[0171] 3) Group sequence number n2, in time order, each group of pictures corresponds to a group sequence number n2, and the group sequence number n2 ranges from 1 to N4, N4 = N1 x l1 x P in this embodiment; l1 represents the number of rounds of picture presentation in the visual evoked stimulation task. Among them, there are d1 EEG samples in the EEG sample set corresponding to each group of pictures, and P represents the total number of visual evoked stimulation tasks.

[0172] 4) Presentation picture sequence number n3, the presented pictures are numbered in time sequence, and the presentation picture sequence number n3 ranges from 1 to N5, N5 = N1 x d1 x l1 x P in this embodiment; l1 represents the number of rounds of picture presentation in the visual evoked stimulation task, and P represents the total number of visual evoked stimulation tasks.

[0173] Step 2-25: Repeat steps 2-22 to 2-24; obtain EEG data samples for each picture presentation in all visual evoked stimulation tasks and labels for EEG samples for each picture presentation; all EEG sample labels for each picture presentation and all EEG samples for each picture presentation form a first EEG sample set. The dimension of all EEG samples for each picture presentation is N5 x number of leads x 1 second data points. The dimension of all EEG sample labels for each picture presentation is N4 x 4.

[0174] And the screening of EEG data using eye movement data mainly includes the following steps:

[0175] Step 2-31: According to the group sequence number n2, all samples in the corresponding group in the first eye movement sample set are extracted, and the samples with target information of 1 are used as the first eye movement temporary sample;

[0176] Step 2-32: Find the corresponding target cell according to the figure position number in the first eye movement temporary sample label, and determine the coordinate range of the target cell. If the data points exceeding the second threshold v2 in the coordinate data of the first eye movement temporary sample are not within the range of the corresponding target cell, eliminate all EEG samples and labels in the first EEG sample set. The value of the second threshold v2 depends on the eye movement data processing method and noise condition, and the second threshold v2 is usually 10%.

[0177] Step 2-33: Group number n2 is incremented by 1, and steps 2-31 to 2-32 are repeated until all groups are traversed. The second EEG sample set is obtained.

[0178] Step 2-34: If the eliminated EEG samples exceed 20% of the total number of EEG samples in the first EEG sample set, it indicates that the data acquisition quality is poor, and the subsequent operation is terminated.

[0179] The EEG data can also be screened in the first round by eye movement data, and a part of low-quality EEG data is eliminated. According to the result of the screening, the second round of screening of the EEG data is performed, and the screening method of the EEG data is the same as that in embodiment 1. The low-quality data processing method is the same, and the effective EEG data is obtained.

[0180] Step 3: Analyze the effective EEG data to obtain the interest degree of the participants in the research object; mainly including the following steps:

[0181] Step 30: Downsample the second EEG sample set to reduce the sampling rate to 50 Hz. The label of the sample is unchanged, and the third EEG sample set is formed;

[0182] Step 31: Shuffle the third EEG sample set, and divide it into N6 different combinations with the group as the minimum unit. The combination number n4 ranges from 1 to N6, and the fourth EEG sample set is formed.

[0183] The number of combinations can be more than or less than 10. The more the combinations, the longer the calculation time, but the higher the accuracy. Among them, the sample combination meets the following requirements:

[0184] 1) The samples with the same group number form a sample group.

[0185] 2) A sample group is in the same combination and can only belong to one combination.

[0186] 3) Each sample group is randomly divided into different combinations.

[0187] 4) All samples in the third EEG sample set are in the fourth EEG sample set, and the number of samples is consistent.

[0188] 5) In each combination, the number of samples in the sample group is consistent or close to consistent.

[0189] Step 32: Start cross-validation, each sample gets a classification value, as the first electroencephalogram classification set. The larger the value, the more likely the sample corresponds to the picture that appears on the target cell.

[0190] The cross-validation steps are as follows:

[0191] Step 321: Extract all electroencephalogram sample sets corresponding to combination sequence n4 from the fourth electroencephalogram sample set, retain the target information in the label of the electroencephalogram sample in the electroencephalogram sample set corresponding to combination sequence n4, form a first electroencephalogram temporary sample set;

[0192] Step 322: Extract all electroencephalogram sample sets except combination sequence n4 from the fourth electroencephalogram sample set, and only retain the target information in the label of the electroencephalogram sample, form a second electroencephalogram temporary sample set;

[0193] Step 323: Train the classifier using the second electroencephalogram temporary sample set, and the trained classifier is the first temporary classifier. Wherein, the classifier can be SVM, neural network or other classifier; In the training process, the electroencephalogram sample in the second electroencephalogram temporary sample set is taken as the input signal, and the corresponding target information is taken as the output to train the parameters in the classifier.

[0194] Step 324: Classify each sample in the first electroencephalogram temporary sample set using the first temporary classifier. For each electroencephalogram sample, the classifier will give a classification value;

[0195] Step 325: n4+1, repeat steps 321-324 until all electroencephalogram samples in the combination get a classification value. The classification value of each electroencephalogram sample and the label information of each electroencephalogram sample are combined to form the first electroencephalogram classification set.

[0196] Step 33: Process the first electroencephalogram classification set to obtain the first electroencephalogram result set.

[0197] A set of electroencephalogram samples should contain d1 electroencephalogram samples, representing pictures appearing on different cells. Only one picture appears on the target cell, and the others appear on non-target cells. The investigator can only observe the picture appearing on the target cell and generate the corresponding electroencephalogram signal. Correspondingly, the classification value of the picture appearing on the target cell should also be the largest in the group.

[0198] Step 331: Take a group of data from the first electroencephalogram classification set as the first electroencephalogram temporary classification set.

[0199] Step 332: Find the picture sequence number corresponding to the sample with target information of 1 in the first electroencephalogram temporary classification set, as the target picture number, and record it.

[0200] Step 333: Find the target information corresponding to the sample with the largest classification value in the first temporary EEG classification set as the classification result of this group, and record it. If the target information corresponding to the sample with the largest classification value is 1, it means that the subject can successfully identify the image of interest through EEG data, otherwise it means that it cannot.

[0201] Step 334: Repeat steps 331-333 until all groups are calculated, and the data recorded in steps 332 and 333 form the first EEG result set.

[0202] Step 34: Determine the interest level of the research participants in the research object based on the first EEG result set and the second EEG sample set. This includes the following steps:

[0203] Step 341: Extract EEG samples with target information of 1 as the first temporary EEG sample in the second EEG sample set according to group number n2. If the group cannot be found, it means that the group has been excluded, and this loop is skipped.

[0204] Step 342: Extract data with classification result of 1 in the first EEG result set according to group number n2. If the group cannot be found, it means that the group classification is incorrect, and this loop is skipped.

[0205] Step 343: Calculate the EEG feature values related to image perception as the EEG feature values corresponding to the target image number. In this embodiment, the EEG feature values obtained include peak feature and energy feature, which are selected from the computer data at the top of the head, the P300 component of the EEG data, and calculated as follows: Calculate the maximum value of the first temporary EEG sample, Cz lead (or other electrodes at the top of the head), within 100-400 milliseconds, as the peak feature. Take the sum of the absolute values of the EEG within the first 100-400 milliseconds as the energy feature, and record it.

[0206] Step 344: Group number n2+1, repeat steps 341-343; until all group data is traversed.

[0207] Step 345: Normalize all feature values by type. Taking the peak feature as an example, the specific calculation steps are as follows: First, calculate the average value and standard deviation of all peak features. Then subtract the average value from the peak data, and then divide by the standard deviation to get the new peak feature. The new energy feature uses the same calculation method as the new peak feature.

[0208] Step 346: For different target image numbers, calculate the average value of the new peak feature and the new energy feature (without distinguishing between feature categories) as the EEG feature value of the image number.

[0209] Step 347: According to the corresponding relationship between the research object and the target figure number, the interest degree of the research personnel to the research object is sorted from high to low according to the electroencephalogram characteristic value.

[0210] The present application also provides a computer system, comprising: one or more processors; a memory storing instructions operable, which when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising the process of the aforementioned research method based on visual evoked stimulation.

[0211] It should be understood that the aforementioned example of the research method based on visual evoked stimulation of the present application can be implemented in any computer system with data storage and data processing, which can be at least one electronic processing system or electronic device comprising a processor and a memory, such as a PC computer, whether it is a personal PC computer, a commercial PC computer, or a graphic processing PC computer, a server-level PC computer. These PC computers can realize wired and / or wireless data transmission, especially image data, through data interfaces and / or network interfaces.

[0212] In other embodiments, the computer system can also be a server, especially a cloud server, with data storage, processing and network communication functions.

[0213] The example computer system generally includes at least one processor, a memory and a network interface connected by a system bus. The network interface is used for communication with other devices / systems.

[0214] The processor is used to provide the calculation and control of the system.

[0215] The memory includes a non-volatile memory and a cache.

[0216] The non-volatile memory usually has a mass storage capacity and can store an operating system and computer programs, which can include instructions operable, which when executed by one or more processors, enable the one or more processors to perform the process of the research method based on visual evoked stimulation of the aforementioned embodiments of the present application.

[0217] In the implementation mode as required or reasonable, the aforementioned computer system, whether it is a PC device or a server, can also include more or less components than those shown in the figure, or combinations, or different hardware, software, etc. Different components or different deployment modes.

Claims

1. A research method based on visually evoked stimuli, characterized in that: Includes the following steps: Electroencephalogram (EEG) data were collected from participants in the visually evoked stimulus task. The collected EEG data is filtered to obtain valid EEG data; Analyze valid EEG data to obtain the level of interest of the survey participants in the survey subjects; The visual evoked stimulus task is as follows: Set one of the cells in the survey observation grid as the target cell, which is used to observe the image; A set of images is presented once in each cell of the survey observation grid. A round of image presentation consists of all images in the image set being presented once in each cell. During each round of image presentation, each image in the image set appears randomly in a cell of the survey observation grid, and only one image appears at a time. Two adjacent images cannot appear in the same cell. Each image in the image set must be presented once in each cell of the survey observation grid. The round of image presentation ends after all images in the image set have been presented. The image set includes both survey subjects and non-survey subjects.

2. The survey method based on visually evoked stimuli according to claim 1, characterized in that: The screening of the collected EEG data includes preprocessing and low-quality data processing. The preprocessing method is as follows: The collected EEG data from the participants in the survey were filtered. Extract the EEG data from the filtered EEG data for each time an image is presented, from 1 second before the image appears to 1 second after the image appears, and obtain the EEG data for each time an image is presented, which will be used as the original EEG sample for each time an image is presented. The EEG data of the first second in the original EEG sample at each image presentation was used as the baseline, and each lead of the original EEG sample at each image presentation was individually calibrated to the baseline. Extract data from the moment the image appears until one second after the image appears within each data segment to form EEG samples for each time the image is presented. Labels were set for the EEG samples each time an image was presented; in the visual evoked stimulus task, the labels of the EEG samples each time an image was presented and the EEG samples each time an image was presented constituted the first EEG sample set; The method for processing low-quality data is as follows: delete samples from the first EEG sample set whose maximum absolute value of the EEG sample is greater than a first threshold each time an image is presented. Valid EEG data were obtained.

3. The survey method based on visually evoked stimuli according to claim 1, characterized in that: It includes multiple visually evoked stimulus tasks, each with a different target cell.

4. The survey method based on visually evoked stimuli according to claim 1, characterized in that: In the visually evoked stimulus task, eye movement data was collected from the participants, and the brain wave data was filtered through the eye movement data to obtain effective brain wave data.

5. The survey method based on visually evoked stimuli according to claim 1, characterized in that: In the visual evoked stimulus task, eye movement data of the participants were collected. The brain wave data was then filtered using the eye movement data to remove some low-quality brain wave data. Based on the results of the initial filtering, the brain wave data was further filtered to obtain the effective brain wave data.

6. The survey method based on visually evoked stimuli according to claim 1, characterized in that: Analyzing valid EEG data involves cross-validation on a classifier to obtain a classification value for each sample. Calculate the average category value for each image presented in the target cell; then, sort the images by their index from highest to lowest based on the average category value to determine the level of interest among the survey participants in the survey subjects.

7. The survey method based on visually evoked stimuli according to claim 6, characterized in that: The cross-validation process involves combining valid EEG data, selecting one combination as the group to be classified, using all other combinations as the training set to train a classifier, and then using the trained classifier to classify each sample in the group to be classified, thus obtaining the classification value for each sample in the group to be classified. This process is repeated for all combinations as the group to be classified, until the classification values ​​for all samples are obtained.

8. The survey method based on visually evoked stimuli according to claim 1, characterized in that: Methods for obtaining the level of interest of survey participants in the survey subjects also include statistically analyzing the feature information of image perception in EEG data as the EEG feature value corresponding to the target image number; sorting the EEG feature values ​​from high to low to obtain the ranking of the subjects' level of interest in the survey subjects; wherein, the feature information includes EEG peak feature and EEG energy feature; the EEG feature value is the average of the EEG peak feature and the EEG energy feature; the peak feature is the maximum value of the EEG signal at the electrode at the top of the head within 100-400 milliseconds; the EEG energy feature is the sum of the absolute values ​​of the EEG samples within 100-400 milliseconds each time the image is presented.

9. A computer-readable medium for storing software, characterized in that, The software includes instructions executable by one or more computers, which, upon execution, cause the one or more computers to perform operations including the flow of a research method based on visually evoked stimuli as described in any one of claims 1-6.

10. A computer system, characterized in that, include: One or more processors; The memory stores operable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, including the flow of a survey method based on visually evoked stimuli as described in any one of claims 1-6.

Images