Brain-computer interface device control method, system, and storage medium

Abstract

The application discloses a brain-computer interface device control method and system and a storage medium. The method comprises the following steps: generating a plurality of test videos associated with a plurality of cover images based on a target original video; wherein each cover image corresponds to at least one test video; determining a cover image focused on by a subject based on the plurality of cover images; determining a test video matched with the subject from the test video corresponding to the cover image focused on by the subject; obtaining a basic combination of the subject according to the cover image focused on by the subject and the test video matched with the subject for each subject; and counting the basic combinations of a plurality of subjects, and determining a recommended combination as a basic combination with the highest occurrence frequency. The application can obtain the area focused on by the subject without collecting eye movement data of the subject, thereby simplifying the data processing process.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a brain-computer interface device control method, system, and storage medium. Background Technology

[0002] Neuromarketing refers to the application of neuroscience methods to study consumer behavior, explore the neural mechanisms underlying consumer decision-making, and identify the true driving forces behind consumer behavior, thereby generating appropriate marketing strategies. It has actually emerged alongside the development of several fundamental disciplines supporting marketing theory in recent years. Among them, the major breakthroughs in cognitive science and neuroscience have played a key role. With the development of behavioral decision-making and cognitive science, marketing theory can borrow many psychological concepts to explain consumer behavior, such as implicit memory, automatic information processing, and the subconscious. Since the human brain controls all aspects of human behavior, understanding how the human brain works not only helps explain human behavior but also helps marketers grasp the patterns of consumer behavior. With the development of brain-computer interface technology, by having subjects wear EEG devices to watch marketing videos, the EEG signals of the subjects are collected and analyzed to determine whether the marketing videos can attract the subjects' attention. Existing technologies disclose several methods for evaluating image information by collecting EEG signals.

[0003] For example, Chinese patent application "CN110432915B" discloses a method and apparatus for evaluating the creativity of an information stream, including: collecting electroencephalogram (EEG) signals and eye-tracking data during the process of a subject browsing a target information stream; determining the start and end times of the subject's gaze in each creative region of the target information stream on a time axis based on the eye-tracking data; calculating the gaze duration for each creative region based on the gaze start and end times; acquiring the corresponding EEG signal of the creative region on a time axis; determining the attentional attraction of the creative region to the subject based on the gaze duration and the browsing duration of the target information stream; determining the emotional arousal and information transmission degree of the corresponding creative region to the subject based on the EEG signal of the creative region; and obtaining the subject's evaluation of the information stream creativity based on the creative weight of each creative region, the attentional attraction, emotional arousal, and information transmission degree of the creative region to the subject.

[0004] However, this method requires additional eye movement data collection from the subjects, and the electroencephalogram (EEG) signals need to be analyzed based on the eye movement data. Therefore, the data processing method of this scheme is relatively complex. Summary of the Invention

[0005] Objective: In view of at least one of the prior art, the present invention provides a brain-computer interface device control method, system and storage medium that can collect the subject's attention without collecting eye movement data.

[0006] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a brain-computer interface device control method, comprising:

[0008] Step S1: Based on the target original video, generate and store multiple test videos associated with multiple cover images; wherein each cover image corresponds to at least one test video;

[0009] Step S2: Based on the multiple cover images, determine the cover image that the subject is interested in;

[0010] Step S3: Identify the test video that matches the subject from the test videos corresponding to the cover image that the subject is interested in;

[0011] Step S4: For each subject, obtain the subject's basic combination based on the cover image that the subject is interested in and the test video that matches the subject;

[0012] Step S5: Analyze the basic combinations of multiple subjects and determine the most frequently occurring basic combination as the recommended combination.

[0013] In some embodiments, step S2: determining the cover image that the subject is interested in based on the plurality of cover images includes:

[0014] Step S21: Extract at least one cover image and embed it into the background image of each display unit in the display area; wherein the number of extracted cover images is the same as the number of display units configured in the display area;

[0015] Step S22: Acquire the EEG signal of the display area viewed by the subject, and determine the cover image that the subject is paying attention to based on the EEG signal; wherein the display area flashes the background image in each display unit at the same first frequency, and flashes the cover image in each display unit at a second frequency different from the first frequency, and the second frequency of each cover image is different.

[0016] In some embodiments, acquiring electroencephalogram (EEG) signals of the display area viewed by the subject, and determining the cover image that the subject is focusing on based on the EEG signals, includes:

[0017] Acquire the subject's electroencephalogram (EEG) signals and extract the sub-frequency and amplitude contained in the EEG signals;

[0018] If the sub-frequency contains both a first value and a second value, and a predetermined amplitude appears at a first moment, then it is determined that the subject is paying attention to the cover image flashing at the second frequency, wherein the first value is an integer multiple of the first frequency, and the second value is an integer multiple of the second frequency.

[0019] If the sub-frequency contains both the first value and the second value, but the preset amplitude does not appear at the first moment, the subject is determined to be in the first state;

[0020] If the sub-frequency contains only the first value, the subject is determined to be in the second state.

[0021] Furthermore, if the subject is determined to be in the first state, the cover image that the subject is looking at is obtained based on the second value and used as a low-attention image;

[0022] Obtain the basic elements included in the low-attention image, and generate an updated cover image based on the basic elements;

[0023] The updated cover image was re-extracted.

[0024] Furthermore, if the subject is determined to be in the second state, other cover images that have not been selected are drawn until a cover image that attracts the subject's attention is obtained, or all cover images have been selected.

[0025] In some embodiments, step S3: determining the test video that matches the subject from the test videos corresponding to the cover image that the subject is interested in, including:

[0026] Step S31: Extract the alpha and beta waves of the EEG signal when the subject watches the test video; wherein the test video watched by the subject is a test video extracted from at least one test video corresponding to the cover image that the subject is paying attention to;

[0027] Step S32: Based on the ratio of β wave to α wave, determine whether the subject matches the test video;

[0028] Step S33: If the subject does not match the test video, repeat steps S31 to S32 until all test videos corresponding to the cover image that the subject is interested in are extracted.

[0029] Step S34: If the subject matches the test video, then the matching success information is recorded, wherein the matching success information includes the cover image that the subject has paid attention to and the test video pair that matches the subject.

[0030] In some embodiments, step S32, determining whether the subject matches the test video based on the ratio of beta wave to alpha wave, includes:

[0031] Obtain the second ratio of β wave and α wave at each acquisition time point when the subject watches the test video. Based on the second ratio and the acquisition time point, obtain the second curve in the acquisition time point-ratio coordinate system. Calculate the enclosed area between the second curve and the first curve in the coordinate system. If the enclosed area is greater than or equal to a first threshold, it is determined that the subject matches the test video.

[0032] The method for obtaining the first curve is as follows: obtain the first ratio of β wave to α wave at each acquisition time point when the subject has not watched the test video, and obtain the first curve in the time point-ratio coordinate system based on the first ratio and the acquisition time point.

[0033] In some embodiments, the brain-computer interface device control method further includes: obtaining the second curve in real time based on the second ratio when the subject is watching the test video; if the encircled area is less than the second threshold when the second time point is reached, adding a stimulus element to the test video until the end of the test video; if the encircled area is greater than the first threshold at this time, incrementing the value of the statistical value of the stimulus element by 1.

[0034] After all subjects have completed the test, if the statistical value of the stimulus element is greater than the third threshold, recommendation information is generated to embed the stimulus element into the test video.

[0035] In a second aspect, the present invention provides a brain-computer interface device control system, comprising:

[0036] The generation module is configured to: generate and store multiple test videos associated with multiple cover images based on the target original video; wherein each cover image corresponds to at least one test video;

[0037] The first determining module is configured to: determine the cover image that the subject is interested in based on the multiple cover images;

[0038] The second determining module is configured to: determine the test video that matches the subject from the test videos corresponding to the cover image that the subject is interested in;

[0039] The basic combination acquisition module is configured to: for each subject, obtain the subject's basic combination based on the cover image that the subject is interested in and the test video that matches the subject;

[0040] The recommended combination determination module is configured to: count the basic combinations of multiple subjects, and determine the basic combination that appears most frequently as the recommended combination.

[0041] Thirdly, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0042] Beneficial Effects: The brain-computer interface device control method, system, and storage medium provided by this invention offer a technical solution for collecting subject attention without the need for eye-tracking data. This invention first generates multiple test videos based on the original video, and then correlates the cover image with the test videos. A background image and a cover image are displayed in the display area, with the cover image embedded in the background image and flashing simultaneously. The subject's EEG signals are acquired and analyzed to determine which cover image the subject is currently focusing on. Next, the test video associated with the subject's currently focused cover image is played, and the subject's EEG signals are acquired and analyzed to determine if the subject matches the test video. If a match is found, the cover image and test video selected by the subject are determined as the basic combination. Finally, the frequency of occurrence of the basic combination is counted, and the most frequent basic combination is recorded as the recommended combination. The recommended combination includes cover images that attract a large audience, and the included test videos that enhance the subject's excitement. This provides a reference for researchers. This invention allows the test to be automatically completed by the subject's EEG-controlled device without the need for manual operation, thus reducing the workload of researchers. This invention allows for automated testing controlled by the subject's electroencephalogram (EEG) without the need for human intervention, thus reducing the workload for researchers. Furthermore, this invention simplifies data processing by identifying the area of ​​focus of the subject without requiring eye-tracking data. Attached Figure Description

[0043] Figure 1 This is a flowchart of a brain-computer interface device control method according to an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the display area according to an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram of the first curve and the second curve according to an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of the structure of a brain-computer interface device control system according to an embodiment of the present invention. Detailed Implementation

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention, and should not be used to limit the scope of protection of the present invention.

[0048] In the description of this invention, "several" means one or more, "multiple" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0049] In the description of this invention, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0050] Example 1

[0051] like Figure 1 As shown, a brain-computer interface device control method includes:

[0052] Step S1: Based on the target original video, generate and store multiple test videos associated with multiple cover images; wherein each cover image corresponds to at least one test video;

[0053] In this embodiment, the original video is an unedited video image; for example, various marketing videos or corporate introduction videos. The original video is cut into multiple video segments, and then multiple video segments are extracted and combined into a test video. Similarly, other test videos are generated based on this principle. After the test video is generated, multiple cover images are set. The cover images can be manually designed posters or static images extracted from the test video. Then, the test video and its corresponding cover image are associated and stored in a storage module, which can be a storage device such as an HDD or SSD.

[0054] In this embodiment, each cover image corresponds to at least one test video; for example, assuming that the storage module stores 12 cover images and 24 test videos, the 12 cover images are numbered A1, A2, B1, B2, ..., F1, F2 respectively, cover image A1 is associated with test videos A1a1 and A1a2, cover image A2 is associated with test videos A2a1 and A2a3, and the rest will not be further elaborated.

[0055] Step S2: Based on the multiple cover images, determine the cover image that the subject is interested in;

[0056] In some embodiments, step S2 may include:

[0057] Step S21: Extract at least one cover image and embed it into the background image of each display unit in the display area; wherein the number of extracted cover images is the same as the number of display units configured in the display area;

[0058] In this embodiment, such as Figure 2 As shown, the display area has a display area P1, which is pre-configured with 6 display units (1-6). Each display unit includes a background image P2. During testing, the display area first randomly selects 6 cover images from the storage module and embeds them into the background image P2, as shown. Figure 2 The cover image P3 is embedded in the background image P2.

[0059] Step S22: Acquire the EEG signal of the display area viewed by the subject, and determine the cover image that the subject is paying attention to based on the EEG signal; wherein the display area flashes the background image in each display unit at the same first frequency, and flashes the cover image in each display unit at a second frequency different from the first frequency, and the second frequency of each cover image is different.

[0060] In this embodiment, the subject's electroencephalogram (EEG) signals can be acquired and analyzed using a monitoring device. Specifically, the monitoring device is an EEG signal acquisition device worn on the subject's head, used to acquire the subject's EEG signals. The EEG signals are then processed using a Fast Fourier Transform (FFT) to obtain the sub-frequency components. Based on these sub-frequency components, the analysis identifies which cover image the subject is focusing on. The specific analysis method will be described later. If the analysis indicates that the subject is focusing on cover image A1, one of the test videos A1a1 and A1a2 is extracted and played. In this embodiment, test video A1a1 is played first. If the analysis indicates that the subject is not focusing on any of the six display units' cover images, the remaining six cover images are extracted from the storage module and displayed in the display area. If, after display, the analysis indicates that the subject is still not focusing on any cover image, extraction stops, and a test completion reminder is generated.

[0061] Further, in some embodiments, step S22, acquiring the electroencephalogram (EEG) signal of the display area viewed by the subject, and determining the cover image that the subject is paying attention to based on the EEG signal, includes:

[0062] Acquire the subject's electroencephalogram (EEG) signals and extract the sub-frequency and amplitude contained in the EEG signals;

[0063] If the sub-frequency contains both a first value and a second value, and a predetermined amplitude appears at a first moment, then it is determined that the subject is paying attention to the cover image flashing at the second frequency, wherein the first value is an integer multiple of the first frequency, and the second value is an integer multiple of the second frequency.

[0064] If the sub-frequency contains both the first value and the second value, but the preset amplitude does not appear at the first moment, the subject is determined to be in the first state;

[0065] If the sub-frequency contains only the first value, the subject is determined to be in the second state.

[0066] First, let's explain the basic principle of this step. Steady-state visual evoked potential (SSVEP) refers to the continuous response of the human brain's visual cortex to a stimulus of a fixed frequency (at the fundamental or harmonic frequency of the stimulus) when stimulated by a visual stimulus of a fixed frequency. This response can be obtained by analyzing the electroencephalogram (EEG) signal. In addition, when an event occurs, if the brain focuses on the event, a positive peak will appear in the EEG signal 300ms after the event occurs. The above principles are widely used and will not be elaborated here.

[0067] Refer again Figure 2 The display area flashes six background images P2 in the image at the same first frequency. The background images P2 are solid color images, and the first frequency is 40 Hz. The display area flashes the cover image at different second frequencies. In this embodiment, the cover image in display unit 1 flashes at a frequency of 10 Hz, and the cover image in display unit 2 flashes at a frequency of 15 Hz. The flashing frequencies of the other cover images are not listed. When the subject views the display area, the subject's EEG signal is analyzed. If frequencies of 40 Hz and 10 Hz, or frequencies that are integer multiples of 40 Hz and 10 Hz, are detected, and a peak is detected 300 ms after the start of viewing, the monitoring device determines that the subject is paying attention to the cover image of display unit 1. If only frequencies of 40 Hz and 10 Hz are detected in the subject's EEG signal, and no peak is detected, the subject is set to the first state, indicating that although the subject is viewing the cover image of display unit 1, the subject is not concentrating on viewing. If only a frequency of 40 Hz is detected in the subject's EEG signal, the subject is set to the second state, indicating that the subject is not viewing any cover image, that is, the subject's attention is completely scattered.

[0068] Furthermore, in this embodiment, if it is determined that the subject is in the first state, the cover image that the subject is looking at is obtained based on the second value and used as a low attention image;

[0069] Obtain the basic elements included in the low-attention image, and generate an updated cover image based on the basic elements;

[0070] The updated cover image was re-extracted.

[0071] Furthermore, if the subject is determined to be in the second state, other cover images that have not been selected are drawn until a cover image that attracts the subject's attention is obtained, or all cover images have been selected.

[0072] In some embodiments, the basic elements included in the low-attention image are obtained, and the cover image is updated based on the basic elements. This can be achieved by calling a pre-configured recognition model. The recognition model can be built based on OpenAI's GPT series models or other large language models. The methods for building and using the recognition model are existing technologies and will not be described in detail here. If the subject is in the first state, the second frequency in the subject's EEG signal is obtained, for example, 10Hz, indicating that the subject is viewing the cover image in display unit 1. The cover image in display unit 1 is defined as a low-attention image, and then the low-attention image is input into the recognition model. The recognition model first recognizes the low-attention image and obtains the basic elements included in the image. In this embodiment, the basic elements include the objects and image style included in the image. Then, the recognition model regenerates the cover image based on the recognized basic elements and randomly embeds it into the background image.

[0073] For example, if the recognition model identifies three objects—road, smoke, and office buildings—in a low-attention image, and the image style is identified as cyberpunk, a style characterized by strong visual contrast, a dark overall tone, and a technological feel, then the model will regenerate a low-attention image that also includes roads, smoke, and office buildings, and retains the cyberpunk style. Next, a cover image is randomly selected from six display units for replacement. For instance, the newly generated cover image replaces the original cover image in display unit 2. Simultaneously, since the remaining five cover images in the display area are not attracting attention, the remaining cover images are retrieved from the storage module and used to replace the currently displayed cover images in the display area. Furthermore, if only 40Hz is detected within a sub-frequency range, it indicates that none of the six cover images in the display area have attracted the subject's attention. Therefore, the remaining six cover images are retrieved from the storage module and used to replace all currently displayed cover images in the display area.

[0074] The beneficial effect of the above steps is that the subject's attention state can be judged based on the subject's EEG signals. The present invention creatively sets a background image and places the cover image within the background image. Therefore, the subject can be divided into a first state and a second state. For the subject in the first state, although their attention is not fully focused, since they have viewed the cover image in the display area, it indicates that the subject may be interested in some elements contained in the cover image. At this time, a new cover image is generated based on the original cover image. The new cover image is similar to the original cover image, so the new cover image may be more attractive to the subject. In addition, after generating a new cover image, cover images are continuously extracted from the storage module. In this way, all the cover images displayed in the display area for the first time are updated, so that images that can attract the subject's attention can be found more quickly.

[0075] If a cover image attracts the attention of the test subject, the corresponding test video is extracted and played, and step S3 is performed.

[0076] Step S3: Identify the test video that matches the subject from the test videos corresponding to the cover image that the subject is interested in;

[0077] In some embodiments, step S3 may include:

[0078] Step S31: Extract the alpha and beta waves of the EEG signal when the subject watches the test video; wherein the test video watched by the subject is a test video extracted from at least one test video corresponding to the cover image that the subject is paying attention to;

[0079] Step S32: Based on the ratio of β wave to α wave, determine whether the subject matches the test video;

[0080] In some embodiments, step S32 may include:

[0081] Obtain the second ratio of β wave and α wave at each acquisition time point when the subject watches the test video. Based on the second ratio and the acquisition time point, obtain the second curve in the acquisition time point-ratio coordinate system. Calculate the enclosed area between the second curve and the first curve in the coordinate system. If the enclosed area is greater than or equal to a first threshold, it is determined that the subject matches the test video.

[0082] The method for obtaining the first curve is as follows: obtain the first ratio of β wave to α wave at each acquisition time point when the subject has not watched the test video, and obtain the first curve in the time point-ratio coordinate system based on the first ratio and the acquisition time point.

[0083] In this embodiment, a data collection point is set every 0.25 seconds. That is, when the subject is not watching or watching the video, the monitoring device collects the alpha and beta wave values ​​of the subject every 0.25 seconds. Then, the first ratio of beta waves to alpha waves is directly calculated, and the curve of the first ratio changing over time is obtained on a coordinate system. As is well known, the larger the beta wave to alpha wave ratio, the more excited the person's mental state. The following is an example to illustrate this step. For instance, if the test video is 40 seconds long, the subject is first asked to remain still for 40 seconds. During this time, they can be asked to look at a pure white wall, and the first curve of the first ratio changing over time within 40 seconds is collected. Figure 3 In L1, a second curve is obtained showing how the second ratio changes over time while the subject watches the test video, such as... Figure 3 In the figure, L2 is used to calculate the enclosed area between L1 and L2, which is the shaded area in the figure. The enclosed area can be obtained by calculus, which is the existing technology. Then, the enclosed area is compared with a preset first threshold. If it is greater than the first threshold, it indicates that the test video can attract the attention of the subject, because the second curve L2 generated after watching the test video has a larger fluctuation compared with the first curve L1 when it is still. Similarly, if the enclosed area is less than the first threshold, it indicates that the test video does not cause fluctuation in the subject's mental state.

[0084] Step S33: If the subject does not match the test video, repeat steps S31 to S32 until all test videos corresponding to the cover image that the subject is interested in are extracted.

[0085] Step S34: If the subject matches the test video, then the matching success information is recorded, wherein the matching success information includes the cover image that the subject has paid attention to and the test video pair that matches the subject.

[0086] In this embodiment, the monitoring device is equipped with a bandpass filter unit. The bandpass filter unit filters out data outside the set frequency range. For example, the frequency band of the alpha wave is between 8 and 13 Hz. By filtering out frequency components outside this frequency band, the value of the alpha wave can be obtained. Similarly, the value of the β wave can also be obtained based on this method. The methods for obtaining the alpha and β waves can be found in the literature in the background art. Then, the ratio of β / α is used to determine whether the subject matches the test video. The specific judgment method will be described later.

[0087] If the subject does not match the test video, for example, if the subject focuses on the cover image A1 and the test video A1a1 does not match the subject, then the test video A1a2 is extracted again and tested to see if it matches the subject. If it still does not match, and there is no test video associated with the cover image A1 in the storage module, then a reminder to end the test is generated.

[0088] Step S4: For each subject, obtain the subject's basic combination based on the cover image that the subject is interested in and the test video that matches the subject;

[0089] In this step, steps S2 to S4 are performed for each subject to obtain the basic combination of multiple subjects.

[0090] For example, in this embodiment, the cover image that the first subject focuses on and the matching test video are A1 and A1a1, respectively. Then the basic combination is A1A1a1. Then the second subject wears the monitoring device and repeats the process from step S2 to step S4 to obtain the basic combinations of multiple subjects.

[0091] Of course, it should be noted that it is also possible that if there is no cover image to focus on for a certain subject, or no matching test video, then the corresponding basic combination will not be generated.

[0092] Step S5: Analyze the basic combinations of multiple subjects and determine the most frequently occurring basic combination as the recommended combination.

[0093] In this step, after all subjects have been tested, the frequency of each basic combination is counted, and the basic combination that appears most frequently is recorded as the recommended combination.

[0094] This invention first sets up multiple cover images and test videos, and links the cover images and test videos together. Then, a background image and a cover image are displayed in a display area. During display, the cover image is embedded in the background image and flashes. The EEG signals of the subject are collected and analyzed by a monitoring device to determine which cover image the subject is focusing on. Next, the test video associated with the cover image being focused on is played in the display area. The monitoring device again collects and analyzes the subject's EEG signals to determine if the subject matches the test video. If they match, the cover image and test video selected by the subject are defined as a basic combination. Finally, the frequency of the basic combination is counted, and the basic combination with the most frequent occurrences is recorded as the recommended combination. The recommended combination includes cover images that can attract the attention of a large number of people, and the test video included can also increase the subject's excitement. This can provide a reference for researchers. This invention can automatically complete the test by the subject's EEG-controlled device without the need for other personnel to operate it, thereby reducing the workload of researchers.

[0095] In addition, the present invention can obtain the area of ​​attention of the subject without collecting the subject's eye movement data, thereby simplifying the data processing process.

[0096] In some embodiments, the brain-computer interface device control method further includes: obtaining the second curve in real time based on the second ratio when the subject is watching the test video; if the encircled area is less than the second threshold when the second time point is reached, adding a stimulus element to the test video until the end of the test video; if the encircled area is greater than the first threshold at this time, incrementing the value of the statistical value of the stimulus element by 1.

[0097] After all subjects have completed the test, if the statistical value of the stimulus element is greater than the third threshold, recommendation information is generated to embed the stimulus element into the test video.

[0098] Specifically, this step is applied during the subject's viewing of the test video. The second time point and the second threshold are less than the test video duration and the first threshold, respectively. For example, if the test video duration is 40 seconds and the first threshold is set to 1, the second time point could be set to 20 seconds and the second threshold to 0.3. During the subject's viewing of the test video, the monitoring mode calculates the area enclosed by the first and second curves in real time. If, at 20 seconds into the test video playback, the area enclosed by the first and second curves is less than 0.3, then it's possible that even after the entire test video has played, the area will not exceed the first threshold. Therefore, to increase the probability of a successful match between the test video and the subject, a stimulus element is added to the test video at this point. At the end of the test, the stimulus element is, for example, various video effects or other pre-set text information. If, after the stimulus element is added and the test video has finished playing, the monitoring device's calculation results show that the test video matches the subject, then the statistical value is incremented by 1. In this embodiment, the third threshold is set to 20. When all subjects have completed the test, if the statistical value is greater than 20, the monitoring device generates recommendation information. This recommendation information includes the following: it is suggested that a stimulus element with a certain name be added to the test video to increase the attention given to the test video. Through this step, the second time point and stimulus element are pre-set in the monitoring device, and the monitoring device can automatically test and statistically analyze whether the stimulus element is effective, thereby reducing the workload of researchers.

[0099] Example 2

[0100] Secondly, based on the above embodiment 1, this embodiment provides a brain-computer interface device control system, including:

[0101] The generation module is configured to: generate and store multiple test videos associated with multiple cover images based on the target original video; wherein each cover image corresponds to at least one test video;

[0102] The first determining module is configured to: determine the cover image that the subject is interested in based on the multiple cover images;

[0103] The second determining module is configured to: determine the test video that matches the subject from the test videos corresponding to the cover image that the subject is interested in;

[0104] The basic combination acquisition module is configured to: for each subject, obtain the subject's basic combination based on the cover image that the subject is interested in and the test video that matches the subject;

[0105] The recommended combination determination module is configured to: count the basic combinations of multiple subjects, and determine the basic combination that appears most frequently as the recommended combination.

[0106] Example 3

[0107] Thirdly, this embodiment provides a storage medium on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0108] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0110] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0112] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for controlling a brain-computer interface device, characterized in that, include: Based on the target original video, generate multiple test videos associated with multiple cover images; Each cover image corresponds to at least one test video; Based on the multiple cover images, the cover image that the subject focused on was determined, including: At least one cover image is extracted and embedded into the background image of each display unit in the display area; wherein the number of extracted cover images is the same as the number of display units configured in the display area; The EEG signals of the subject viewing the display area are acquired, and the cover image that the subject is focusing on is determined based on the EEG signals; wherein the display area flashes the background image in each display unit at the same first frequency, and flashes the cover image in each display unit at a second frequency different from the first frequency, and the second frequency of each cover image is different; From the test videos corresponding to the cover images that the test subjects paid attention to, identify the test videos that matched the test subjects, including: Step S31: Extract the alpha and beta waves of the EEG signal when the subject watches the test video; wherein the test video watched by the subject is a test video extracted from at least one test video corresponding to the cover image that the subject is paying attention to; Step S32: Determine whether the subject matches the test video based on the ratio of beta waves to alpha waves, including: obtaining a second ratio of beta waves to alpha waves at each acquisition time point when the subject watches the test video; obtaining a second curve in the acquisition time point-ratio coordinate system based on the second ratio and the acquisition time point; calculating the enclosed area between the second curve and the first curve in the coordinate system; if the enclosed area is greater than or equal to a first threshold, then the subject matches the test video; wherein the method for obtaining the first curve is: obtaining a first ratio of beta waves to alpha waves at each acquisition time point when the subject has not watched the test video; obtaining a first curve in the time point-ratio coordinate system based on the first ratio and the acquisition time point; when the subject watches the test video, obtaining the second curve in real time based on the second ratio; if the enclosed area is less than the second threshold at the second time point, adding a stimulus element to the test video until the end of the test video; if the enclosed area is greater than the first threshold at this time, incrementing the statistical value of the stimulus element by 1; after all subjects have completed the test, if the statistical value of the stimulus element is greater than a third threshold, generating recommendation information to embed the stimulus element into the test video; Step S33: If the subject does not match the test video, repeat steps S31 to S32 until all test videos corresponding to the cover image that the subject is interested in are extracted. Step S34: If the subject matches the test video, then the matching success information is counted, wherein the matching success information includes the cover image that the subject has paid attention to and the test video pair that matches the subject. For each subject, a basic combination is obtained based on the cover image that the subject pays attention to and the test video that matches the subject. The most frequent basic combination among multiple subjects was selected as the recommended combination.

2. The brain-computer interface device control method according to claim 1, characterized in that, Acquiring electroencephalogram (EEG) signals from the display area viewed by the subject, and determining the cover image that the subject is focusing on based on the EEG signals, including: Acquire the subject's electroencephalogram (EEG) signals and extract the sub-frequency and amplitude contained in the EEG signals; If the sub-frequency contains both a first value and a second value, and a predetermined amplitude appears at a first moment, then it is determined that the subject is paying attention to the cover image flashing at the second frequency, wherein the first value is an integer multiple of the first frequency, and the second value is an integer multiple of the second frequency. If the sub-frequency contains both the first value and the second value, but the predetermined amplitude does not appear at the first moment, the subject is determined to be in the first state; If the sub-frequency contains only the first value, the subject is determined to be in the second state.

3. The brain-computer interface device control method according to claim 2, characterized in that, If the subject is determined to be in the first state, the cover image that the subject is looking at is obtained based on the second value and used as the low attention image; Obtain the basic elements included in the low-attention image, and generate an updated cover image based on the basic elements; The updated cover image was re-extracted.

4. The brain-computer interface device control method according to claim 2, characterized in that, If the subject is determined to be in the second state, other cover images that have not been selected are drawn until a cover image that attracts the subject's attention is obtained, or all cover images have been drawn.

5. A brain-computer interface device control system, characterized in that, include: The generation module is configured to generate and store multiple test videos associated with multiple cover images based on the target original video; Each cover image corresponds to at least one test video; The first determining module is configured to: determine the cover image that the subject is interested in based on the plurality of cover images, including: extracting at least one cover image and embedding it into the background image of each display unit in the display area; wherein the number of extracted cover images is the same as the number of display units configured in the display area; acquiring the electroencephalogram (EEG) signal of the subject viewing the display area, and determining the cover image that the subject is interested in based on the EEG signal; wherein the display area flashes the background images in each display unit at the same first frequency, and flashes the cover images in each display unit at a second frequency different from the first frequency, and the second frequency of each cover image is different; The second determining module is configured to: determine the test video that matches the subject from the test videos corresponding to the cover image that the subject has paid attention to, specifically including: Step S31: Extract the alpha and beta waves of the EEG signal when the subject watches the test video; wherein the test video watched by the subject is a test video extracted from at least one test video corresponding to the cover image that the subject is paying attention to; Step S32: Determine whether the subject matches the test video based on the ratio of beta waves to alpha waves, including: obtaining a second ratio of beta waves to alpha waves at each acquisition time point when the subject watches the test video; obtaining a second curve in the acquisition time point-ratio coordinate system based on the second ratio and the acquisition time point; calculating the enclosed area between the second curve and the first curve in the coordinate system; if the enclosed area is greater than or equal to a first threshold, then the subject matches the test video; wherein the method for obtaining the first curve is: obtaining a first ratio of beta waves to alpha waves at each acquisition time point when the subject has not watched the test video; obtaining a first curve in the time point-ratio coordinate system based on the first ratio and the acquisition time point; when the subject watches the test video, obtaining the second curve in real time based on the second ratio; if the enclosed area is less than the second threshold at the second time point, adding a stimulus element to the test video until the end of the test video; if the enclosed area is greater than the first threshold at this time, incrementing the statistical value of the stimulus element by 1; after all subjects have completed the test, if the statistical value of the stimulus element is greater than a third threshold, generating recommendation information to embed the stimulus element into the test video; Step S33: If the subject does not match the test video, repeat steps S31 to S32 until all test videos corresponding to the cover image that the subject is interested in are extracted. Step S34: If the subject matches the test video, then the matching success information is counted, wherein the matching success information includes the cover image that the subject has paid attention to and the test video pair that matches the subject. The basic combination acquisition module is configured to: for each subject, obtain the subject's basic combination based on the cover image that the subject is interested in and the test video that matches the subject; The recommended combination determination module is configured to: count the basic combinations of multiple subjects, and determine the basic combination that appears most frequently as the recommended combination.

6. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images