Display content control method, device and system based on brain-computer interface

Abstract

The application relates to a display content control method, device and system based on a brain-computer interface, computer equipment, a storage medium and a computer program product. The method comprises the following steps: displaying to-be-displayed content in a content display area of a display interface based on a current display frequency; collecting brain electrical signals of a user in the process in which the user reads the content displayed by the display interface; determining a current reading state of the user based on the collected brain electrical signals and the current display frequency; determining a target display frequency based on the current reading state and a frequency updating strategy, taking the target display frequency as the current display frequency, and performing the step of displaying to-be-displayed content in the content display area of the display interface based on the current display frequency. The method can reduce the number of user operations, improve reading efficiency and reading convenience, and can automatically update the display frequency according to the reading state of the user to adapt to the reading demand of the user.

Description

Technical Field

[0001] This application relates to the field of brain-computer interface technology, and in particular to a method, apparatus, system, computer device, storage medium, and computer program product for controlling display content based on a brain-computer interface. Background Technology

[0002] Existing human-computer interaction methods for e-readers mainly include keyboard and mouse interaction and gesture interaction, which are not suitable for special groups (such as people with hand disabilities). With the development of brain-computer interface technology, human-computer interaction methods based on electroencephalogram (EEG) signals have emerged. By displaying visual stimuli corresponding to various operation commands (such as stimulation areas flashing at different frequencies), when the user looks at different visual stimuli, corresponding EEG signals are generated. By analyzing the EEG signals, the target visual stimulus being looked at by the user can be identified, and the operation command corresponding to that target visual stimulus can be sent to the computer device to execute the corresponding operation, thus realizing human-computer interaction.

[0003] In related technologies, when users read content using a reader, they usually need to manually turn pages frequently, which is inefficient. Summary of the Invention

[0004] Therefore, it is necessary to provide a brain-computer interface-based method, device, system, computer equipment, computer-readable storage medium, and computer program product that can reduce the number of user operations and improve reading efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for controlling display content based on a brain-computer interface. The method includes:

[0006] Based on the current display frequency, display the content to be displayed in the content display area of ​​the display interface;

[0007] While the user is reading the content displayed on the interface, the user's electroencephalogram (EEG) signals are collected;

[0008] Based on the collected EEG signals and the current display frequency, the user's current reading state is determined;

[0009] Based on the current reading state and frequency update strategy, a target display frequency is determined, the target display frequency is used as the current display frequency, and the step of displaying the content to be displayed in the content display area of ​​the display interface based on the current display frequency is executed.

[0010] In one embodiment, determining the user's current reading state based on the acquired EEG signals and the current display frequency includes:

[0011] Based on the current display frequency, multiple filter frequency bands are determined, and the collected EEG signals are filtered based on the multiple filter frequency bands to obtain frequency band signals corresponding to each filter frequency band.

[0012] The canonical correlation analysis algorithm is used to calculate the sub-correlation coefficient between each frequency band signal and the current display frequency, and the correlation coefficient between the EEG signal and the current display frequency is determined based on each sub-correlation coefficient;

[0013] Based on the correlation coefficient, the user's current reading status is determined.

[0014] In one embodiment, determining the user's current reading status based on the correlation coefficient includes:

[0015] If the correlation coefficient is greater than a preset threshold, the user's current reading state is determined to be a concentrated reading state;

[0016] If the correlation coefficient is less than or equal to a preset threshold, the user's current reading state is determined to be a scattered reading state.

[0017] In one embodiment, determining the target display frequency based on the current reading state and the frequency update strategy includes:

[0018] If the user's current reading state is focused reading, the current display frequency is determined as the target display frequency;

[0019] If the user's current reading state is a scattered reading state, the current display frequency is reduced based on the frequency reduction strategy to obtain the target display frequency.

[0020] In one embodiment, the method further includes:

[0021] In the control display area of ​​the display interface, visual stimulus objects corresponding to multiple operation controls are displayed, and each visual stimulus object flashes according to a different code;

[0022] During the process of the user gazing at the control display area, the user's electroencephalogram (EEG) signals are collected, and the collected EEG signals are analyzed and processed to identify the target visual stimulus object that the user is gazing at.

[0023] The target operation is executed based on the operation control corresponding to the target visual stimulus object.

[0024] Secondly, this application also provides a brain-computer interface-based display content control system. The system includes a terminal, an electroencephalogram (EEG) acquisition device, and a data processing device, wherein:

[0025] The terminal is used to display content to be displayed in the content display area of ​​the display interface based on the current display frequency;

[0026] The EEG acquisition device is used to acquire the user's EEG signals while the user is reading the content displayed on the display interface;

[0027] The data processing device is used to determine the user's current reading state based on the acquired EEG signal and the current display frequency; determine a target display frequency based on the current reading state and a frequency update strategy; and send the target display frequency as the current display frequency to the terminal.

[0028] Thirdly, this application also provides a display content control device based on a brain-computer interface. The device includes:

[0029] The display module is used to display the content to be displayed in the content display area of ​​the display interface based on the current display frequency;

[0030] The acquisition module is used to acquire the user's electroencephalogram (EEG) signals while the user is reading the content displayed on the display interface;

[0031] The first determining module is used to determine the user's current reading state based on the collected EEG signals and the current display frequency;

[0032] The second determining module is used to determine the target display frequency based on the current reading state and frequency update strategy, use the target display frequency as the current display frequency, and execute the step of displaying the content to be displayed in the content display area of ​​the display interface based on the current display frequency.

[0033] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in this aspect.

[0034] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in this aspect.

[0035] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in this aspect.

[0036] The aforementioned brain-computer interface-based display content control method, device, system, computer equipment, storage medium, and computer program product display content to be displayed based on the current display frequency. During the user's reading process, the method collects the user's electroencephalogram (EEG) signals and determines the user's current reading state based on the correlation between the EEG signals and the current display frequency. The display frequency is then updated accordingly. This method automatically displays content according to the display frequency, avoiding frequent manual page turning and thus improving reading efficiency and convenience. Furthermore, by analyzing the correlation between the user's EEG signals and the current display frequency, the method determines the user's reading state. Different reading states reflect the compatibility between the current display frequency and the user's reading speed, allowing the display frequency to be automatically updated to match the user's current needs. This eliminates the need for manual adjustments and further improves reading efficiency and convenience. Attached Figure Description

[0037] Figure 1 Here is a schematic diagram of the structure of a content control system as an example;

[0038] Figure 2 This is a flowchart illustrating a brain-computer interface-based display content control method in one embodiment.

[0039] Figure 3 This is a flowchart illustrating the process of determining the current reading state in one embodiment;

[0040] Figure 4 This is a flowchart illustrating a brain-computer interface-based display content control method in another embodiment;

[0041] Figure 5 This is a schematic diagram showing the interface in an example;

[0042] Figure 6 This is a structural block diagram of a brain-computer interface-based display content control device in one embodiment;

[0043] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] First, before introducing the technical solutions of the embodiments of this application, the technical background or evolution of the embodiments of this application will be introduced. Existing human-computer interaction methods for e-readers mainly include keyboard and mouse interaction and gesture interaction, which are not suitable for specific groups (such as some people with disabilities). With the development of brain-computer interface technology, human-computer interaction methods based on electroencephalogram (EEG) signals have emerged. Brain-computer interface technology can directly output signals related to brain activity to external devices without relying on peripheral nerves and muscles. By displaying visual stimuli corresponding to multiple operation commands (such as multiple stimulation area blocks flashing at different frequencies), when the user gazes at different visual stimuli, corresponding EEG signals are generated. By analyzing the EEG signals, the target visual stimulus object gazed at by the user can be identified, and the operation command corresponding to the target visual stimulus object can be sent to the computer device to execute the corresponding operation, thus realizing human-computer interaction.

[0046] Augmented Reality (AR) and Virtual Reality (VR) are two main methods for constructing a metaverse. AR is a technology that maps real-world images onto the environment by calculating the surroundings in real time using a camera. VR, on the other hand, calculates the virtual environment in real time based on the user's movements within a pre-built virtual scene. Currently, the main interaction methods for metaverse scenes built within AR or VR environments are controller interaction, gesture interaction, and voice interaction. These methods either have privacy issues (such as voice interaction) or are not user-friendly for certain groups (such as gesture and controller interaction), and it is difficult to achieve natural and smooth page-turning control during reading, affecting the immersive experience.

[0047] Against this backdrop, through long-term research and development and experimental verification, the applicant proposes a brain-computer interface-based display content control method, which avoids frequent manual page turning, improves reading efficiency and convenience, and features good privacy and user-friendliness for special groups (such as people with hand disabilities). Furthermore, it can automatically adjust the display frequency by monitoring the user's reading status to adapt to the user's reading needs. It should also be noted that the applicant has devoted considerable creative effort to discovering the technical problem and the technical solution described in the following embodiments.

[0048] The brain-computer interface-based display content control method provided in this application can be applied to, for example... Figure 1The diagram shows a display content control system 100. The display content control system 100 includes a terminal 102, a data processing device 104, and an EEG acquisition device 106. The data processing device 104 is communicatively connected to both the terminal 102 and the EEG acquisition device 106. The EEG acquisition device 106 can be mounted on the user's head to collect the user's EEG signals and transmit the collected EEG signals to the data processing device 104 for analysis and processing to determine the display frequency or other operation commands. The data processing device 104 can transmit the determined display frequency or other operation commands to the terminal 102, causing the terminal to display content or perform corresponding operations at that display frequency. The terminal 102 can be, but is not limited to, various electronic devices with a display interface, such as personal computers, laptops, smartphones, tablets, e-readers, and portable wearable devices (such as VR / AR headsets). The terminal 102, data processing device 104, and EEG acquisition device 106 can be independent devices or integrated into one unit (e.g., integrated into a portable wearable device). Furthermore, the data processing device 104 can be deployed locally or in the cloud.

[0049] In one embodiment, such as Figure 2 As shown, a brain-computer interface-based method for controlling display content is provided, which can be applied to... Figure 1 The display content control system shown. In this embodiment, the method includes the following steps:

[0050] Step 201: Based on the current display frequency, display the content to be displayed in the content display area of ​​the display interface.

[0051] In implementation, the terminal of the content display control system can display the content to be displayed in the content display area of ​​the display interface based on the current display frequency. The content to be displayed can be determined according to the reading text selected by the user. The current display frequency refers to the frequency or speed at which the displayed content is updated. For example, if the terminal's content display area can display n characters, the reading text can be extracted in text order, extracting n consecutive characters to obtain multiple sub-texts containing n characters each. These sub-texts can then be concatenated to obtain the complete reading text. Each sub-text is the content to be displayed. The sub-texts are displayed sequentially, and the switching time between two sub-texts corresponds to the display frequency, or display speed. The shorter the switching time, the higher the display frequency (the higher the display speed). The current display frequency can be set by the user or automatically set according to other rules, typically between 3-15 Hz. The number of characters in each sub-text can be matched and set according to the terminal's display parameters (such as display area, font size, etc.). This allows for text display in a fast-sequence visual presentation manner. Rapid sequence visual presentation is a method of quickly presenting content in the same location, which can reduce the frequency of eye movements during reading. It also automatically switches subtexts at a certain display frequency, which can reduce page turning operations during reading.

[0052] Optionally, the content to be displayed can be presented using either a streaming fast sequence visual presentation or a ring-shaped fast sequence visual presentation. In the streaming fast sequence visual presentation method, the content is displayed only in the center of the content display area, with different content appearing sequentially in the center position over time and disappearing after the time limit is met. In the ring-shaped fast sequence visual presentation method, multiple sets of characters are presented simultaneously, appearing sequentially in the center of the display area from right to left and disappearing sequentially at the far left.

[0053] Step 202: Collect the user's EEG signals while the user is reading the content displayed on the interface.

[0054] In practice, users can read the content displayed on the terminal interface, and during this reading process, the EEG acquisition device can collect the user's EEG signals in real time via electrodes. Optionally, the EEG acquisition device can amplify and process the raw EEG signals, and then transmit the processed EEG signals as the collected EEG signals to the data processing device.

[0055] Step 203: Determine the user's current reading state based on the collected EEG signals and the current display frequency.

[0056] In practice, the data processing device can analyze the collected EEG signals based on the terminal's current display frequency to determine the correlation between the EEG signals and the current display frequency. The magnitude of the correlation reflects the user's level of concentration while reading. The data processing device can then identify the user's current reading state based on the magnitude of the correlation.

[0057] Step 204: Based on the current reading status and frequency update strategy, determine the target display frequency, use the target display frequency as the current display frequency, and execute the step of displaying the content to be displayed in the content display area of ​​the display interface based on the current display frequency.

[0058] In implementation, after determining the user's current reading state, the data processing device can determine the target display frequency based on the current reading state and the frequency update strategy, and use the target display frequency as the current display frequency before returning to step 201. For example, the current reading state can include a concentrated reading state and a dispersed reading state. If it is a concentrated reading state, the current display frequency can be kept unchanged; if it is a dispersed reading state, the display frequency can be reduced to adapt to the user's current reading needs.

[0059] In the aforementioned brain-computer interface-based content control method, content is displayed using a rapid sequence visual presentation based on the current display frequency. During the user's reading process, the system collects the user's electroencephalogram (EEG) signals and analyzes the correlation between the EEG signals and the current display frequency to identify the user's current reading state. The display frequency is then updated accordingly. This method automatically switches displayed content based on the display frequency, reducing the number of times the user manually turns pages, thus improving reading efficiency and convenience. Furthermore, by analyzing the correlation between the user's EEG signals and the current display frequency, the system determines the user's reading state. Different reading states reflect the compatibility between the current display frequency and the user's reading speed, allowing for automatic updates to the display frequency to match the user's current needs. This avoids manual adjustments by the user, further improving reading efficiency and convenience. Moreover, when applied to the Metaverse reader scenario, this brain-computer interface interaction method offers advantages such as privacy, user-friendliness for special groups, and the elimination of frequent operations, contributing to an immersive user experience.

[0060] In one embodiment, such as Figure 3 As shown, the process of determining the user's current reading state in step 203 specifically includes the following steps:

[0061] Step 301: Based on the current display frequency, determine multiple filter frequency bands, and based on the multiple filter frequency bands, filter the acquired EEG signals to obtain frequency band signals corresponding to each filter frequency band.

[0062] In practice, the data processing device can determine the starting frequencies of multiple filter bands based on the current display frequency. The calculation formula is as follows:

[0063]

[0064] Where fre is the current display frequency. i represents the filter band number, a positive integer from 1 to N, where N is the total number of filter bands (i.e., the maximum value of i, such as 5). The frequency contained in filter band i is F. i Up to 90 Hz.

[0065] Then, the data processing device can use multiple filters, the same number as the number of filtering frequency bands, to filter the acquired EEG signals and obtain frequency band signals corresponding to each filtering frequency band.

[0066] Step 302: Using canonical correlation analysis algorithm, calculate the sub-correlation coefficient between each frequency band signal and the current display frequency, and determine the correlation coefficient between the EEG signal and the current display frequency based on each sub-correlation coefficient.

[0067] In implementation, for each frequency band signal, the data processing device can use canonical correlation analysis to calculate the sub-correlation coefficient between the frequency band signal and the current display frequency, that is, to extract the steady-state visual evoked potential signal features related to the current display frequency. Then, the data processing device can calculate the correlation coefficient between the EEG signal and the current display frequency based on each sub-correlation coefficient. In one example, the correlation coefficient between the EEG signal and the current display frequency (which can be denoted as...) The following formula can be used to calculate:

[0068]

[0069] in, It is the sub-correlation coefficient between frequency band signal i (or filtered frequency band i) and the current display frequency. The weights corresponding to frequency band signal i can be determined using the formula... The calculations show that a and b are relevant parameters that can be obtained from experiments or experience.

[0070] Step 303: Determine the user's current reading status based on the correlation coefficient.

[0071] During implementation, the data processing device calculates the correlation coefficient. Then, the user's current reading state can be identified based on the correlation coefficient. The correlation coefficient is positively correlated with the user's reading concentration. The larger the correlation coefficient, the more concentrated the user's reading state; the smaller the correlation coefficient, the more dispersed the user's reading state.

[0072] In this embodiment, by determining the filtering frequency band based on the display frequency, the EEG signal is filtered using multi-band filtering to obtain multiple frequency band signals. Then, sub-correlation coefficients are analyzed from each frequency band signal, and the correlation coefficient corresponding to that EEG signal is calculated based on these sub-correlation coefficients. This reflects the compatibility between the user's reading state and the current display frequency. A higher correlation indicates a better compatibility between the user's current reading state and the current display frequency. Conversely, a lower correlation indicates that the current display frequency is not suitable for the user's current reading state, allowing the display frequency to be automatically adjusted according to the current reading state to meet the user's reading needs.

[0073] In one embodiment, step 303, which involves determining the current reading state based on the correlation coefficient, specifically includes the following steps: if the correlation coefficient is greater than a preset threshold, the user's current reading state is determined to be a concentrated reading state; if the correlation coefficient is less than or equal to the preset threshold, the user's current reading state is determined to be a dispersed reading state.

[0074] In practice, the data processing device compares the correlation coefficient between the calculated EEG signal and the current display frequency with a preset threshold. If the correlation coefficient is greater than the preset threshold, the user's current reading state is determined to be a focused reading state; if it is less than or equal to the preset threshold, the current reading state is determined to be a distributed reading state. The preset threshold can be set as needed, such as 0.5.

[0075] In this embodiment, by comparing the correlation coefficient with a preset threshold, the user's current reading state can be identified quickly and accurately. This information can then be used to update the display frequency based on the reading state to adapt to the user's reading needs.

[0076] In one embodiment, step 204, which determines the target display frequency based on the current reading state and the frequency update strategy, specifically includes the following steps: when the user's current reading state is a concentrated reading state, the current display frequency is determined as the target display frequency; when the user's current reading state is a dispersed reading state, the current display frequency is reduced based on the frequency reduction strategy to obtain the target display frequency.

[0077] In implementation, if the data processing device determines that the user's current reading state is concentrated reading, the current display frequency can be set as the target display frequency, i.e., the display frequency remains unchanged. If the current reading state is dispersed reading, the current display frequency can be reduced, such as by reducing it by a preset percentage or a preset value. Optionally, if the reduced target display frequency is less than a preset minimum display frequency (e.g., 3 Hz), the minimum display frequency can be set as the target display frequency, and a prompt message can be displayed on the terminal to remind the user whether to pause and rest.

[0078] In one embodiment, the user can also actively interact with the terminal based on the brain-computer interface, such as... Figure 4 As shown, the method also includes the following steps:

[0079] Step 401: In the control display area of ​​the display interface, display the visual stimulus objects corresponding to multiple operation controls, and each visual stimulus object flashes according to a different code.

[0080] In implementation, the terminal's display interface also includes a control display area, which can flash multiple visual stimuli corresponding to the operation controls according to different codes. When the user gazes at each visual stimulus, corresponding electroencephalogram (EEG) signals are generated. Each visual stimulus can be encoded using steady-state visual evoked potentials (SSVEPs) or event-related potentials (ERPs). In one example, the display interface is as follows: Figure 5 As shown, the display area includes a content display area and a control display area. The control display area displays four operation controls, including a "speed+" control to increase the display frequency, a "speed-" control to decrease the display frequency, a start / pause toggle control, and an exit control.

[0081] Step 402: During the process of the user gazing at the control display area, the user's EEG signal is collected, and the collected EEG signal is analyzed and processed to identify the target visual stimulus object that the user is gazing at.

[0082] During implementation, while the user is gazing at the control display area, the EEG acquisition device can collect the user's EEG signals and analyze them. The analysis process can employ Canonical Correlation Analysis (CCA) to analyze the correlation between the collected EEG signals and various visual stimuli, and output the target visual stimuli that meet the correlation criteria, i.e., identify the operation controls or commands that the user intends to operate.

[0083] In one example, each visual stimulus can be encoded using Event-Related Potentials (ERPs) and classified using a pre-trained Support Vector Machine (SVM) classification model to identify the target visual stimulus corresponding to the EEG signal. Specifically, sample visual stimuli (such as a visual stimulus that changes from black to bright for 150ms and then back to dark at random times) can be displayed using preset visual stimulus codes. While the user is looking at the sample visual stimulus, the user's EEG signal is collected. Then, based on the time stamp of the sample visual stimulus, the EEG signal from 0 to 500ms is extracted as sample data for training the SVM classification model.

[0084] When the user gazes at the visual stimuli corresponding to the various operation controls displayed on the terminal, the EEG acquisition device can collect the user's EEG signals and, based on the stimulation time markers of the visual stimuli, extract a preset number (e.g., 15 segments) of local EEG signals of preset duration (e.g., 0-500ms) from the EEG signals. Then, each local EEG signal is input into an SVM classification model for classification, obtaining the probability value of each visual stimuli corresponding to each local EEG signal. The visual stimuli with the highest probability value are selected as candidate visual stimuli. Then, the probability values ​​of the candidate visual stimuli from each local EEG signal are averaged to obtain the average value. If all candidate visual stimuli are the same, and the average value is greater than a preset threshold (e.g., 0.75), then the candidate visual stimuli can be identified as the target visual stimuli for the user's gaze, and the corresponding operation command can be output to the terminal. Optionally, before inputting each local EEG signal into the SVM classification model, it can be preprocessed, including downsampling the EEG signal, such as downsampling to 100Hz, then performing 50Hz notch filtering, and then performing 1.5-30Hz bandpass filtering, etc.

[0085] Understandably, during the user's use of the reader, the system can continuously collect the user's EEG signals and analyze and process the collected EEG signals. On the one hand, it can identify whether the user is looking at the visual stimulus object corresponding to each operation control. On the other hand, if the user is not looking at the stimulus object corresponding to the operation control, it can identify the user's current reading state so as to automatically update the current display frequency according to the user's reading state.

[0086] Step 403: Execute the target operation based on the operation control corresponding to the target visual stimulus object.

[0087] In practice, after the data processing device identifies the target visual stimulus that the user is looking at, it can send operation instructions to the terminal for the corresponding control, enabling the terminal to perform the target operation. For example, if the data processing device recognizes that the user is looking at... Figure 5The visual stimulus corresponding to the "Exit" control shown can output an exit command to the terminal, causing the terminal to exit the current display interface.

[0088] In this embodiment, the terminal's display interface can also display visual stimuli corresponding to multiple operation controls. Users can then output operation commands to the terminal based on the brain-computer interface by focusing on the visual stimuli, thus achieving human-computer interaction. This approach offers advantages in terms of privacy and is user-friendly for special groups. In the usage scenario of the Metaverse Reader, it is more conducive to the user's immersive experience.

[0089] In one example, step 402, which involves parsing and processing the EEG signals to determine the target visual stimulus object that the user is fixating on, specifically includes the following steps:

[0090] Step 1: Obtain local EEG signals within multiple time windows from the collected EEG signals.

[0091] Among them, the end time of each time window is the same, and the length of each time window is different, and the length difference between two adjacent time windows meets the preset conditions.

[0092] In implementation, the data processing device can acquire local EEG signals within multiple time windows from the acquired EEG signals. Each time window has the same end time; for example, it can all be the last time of the currently acquired EEG signal. The lengths of each time window are different, and the length difference between two adjacent time windows meets a preset condition; for example, the length difference between two adjacent time windows can both be less than a preset value for the minimum time window length. Specifically, the minimum time window length can be set to L seconds (which can be set to 1 to 3 seconds as needed), the length difference between adjacent time windows can be set to f seconds (e.g., 0.2 seconds), and the number of time windows can be set to Q (e.g., 3 to 5). Then, the lengths of the Q time windows are L seconds, (L+f) seconds, and (L+(Q-1)*f) seconds, respectively. The data processing device can extract the EEG signals corresponding to each time window length from the currently acquired EEG signals, starting from the EEG signal at the last time point and proceeding backwards, to obtain Q local EEG signals. Optionally, the length difference between two adjacent time windows can be set to 5% to 20% of the minimum time window length to balance recognition accuracy and efficiency.

[0093] Optionally, before acquiring local EEG signals, the raw EEG signals can be preprocessed, such as downsampling the data to 250Hz and applying a 50Hz notch filter. Alternatively, local EEG signals can be acquired first, and then preprocessed for each local EEG signal.

[0094] Step 2: Determine multiple filter frequency bands based on the frequency information corresponding to each visual stimulus.

[0095] In implementation, the visual stimuli corresponding to the multiple operation controls displayed on the terminal's screen can be encoded based on steady-state visual evoked potentials. The frequency range of each visual stimulus can be set to 16 to 17.5 Hz, with a frequency interval of 0.5 Hz, and the filter bands can be set to three: 14-90 Hz, 30-90 Hz, and 46-90 Hz.

[0096] Optionally, each visual stimulus can be encoded using a frequency-phase joint encoding method, and the brightness of the stimulus changes over time as shown in the following formula:

[0097]

[0098] Where frequency represents the frequency of each visual stimulus. This is the initial phase. For example... Figure 5 The visual stimuli corresponding to the four control buttons shown can have their frequencies and phases set as follows: Exit button: 16 Hz, Phase 0; Start / Pause switch button: 17 Hz, Phase 0.5π; Speed+ button: 16.5 Hz, Phase 0; Speed- button: 17.5 Hz, Phase π. Refreshrate refers to the number of times the stimulator refreshes the screen content per second, usually expressed in Hertz (Hz). For example, a 60Hz refresh rate means the monitor refreshes the stimulus 60 times per second. 'i' indicates the i-th refresh of the stimulator content according to the refresh rate.

[0099] Step 3: For each local EEG signal, filter the local EEG signal based on multiple filter frequency bands to obtain the frequency band signal corresponding to each filter frequency band.

[0100] In practice, the data processing device can use multiple filters to perform multi-band filtering on each local EEG signal to obtain frequency band signals of multiple filtered frequency bands corresponding to each local EEG signal.

[0101] Step 4: Using canonical correlation analysis algorithm, calculate the sub-correlation coefficient between each frequency band signal and each visual stimulus object, and calculate the correlation coefficient between the local EEG signal and each visual stimulus object based on each sub-correlation coefficient.

[0102] In practice, for each local EEG signal, the data processing device can calculate the sub-correlation coefficient between each frequency band of the local EEG signal and each visual stimulus, that is, it can extract the steady-state visual evoked potential signal features (SSVEP signal features) corresponding to the frequencies of each visual stimulus in each frequency band signal. Then, the data processing device can calculate the correlation coefficient between the local EEG signal and each visual stimulus based on each sub-correlation coefficient and the weights corresponding to each frequency band signal. In one example, the correlation coefficient between local EEG signal j and visual stimulus k (which can be denoted as...) The calculation formula for ) is as follows:

[0103]

[0104] in, This represents the sub-correlation coefficient between the i-th frequency band signal and the visual stimulus k. k is a positive integer from 1 to N, where N represents the total number of visual stimuli, the same as the maximum value of k. The weights corresponding to the i-th frequency band signal can be represented by the formula. The calculation yields the result, where a and b are relevant parameters that can be obtained through experiments or experience. i is a positive integer from 1 to M, where M represents the total number of signals in the frequency band, and is the same as the maximum value of i.

[0105] Step 5: Based on the correlation coefficient between the local EEG signal and each visual stimulus object, the visual stimulus object with the highest correlation coefficient is determined as the initial visual stimulus object corresponding to the local EEG signal.

[0106] In practice, the data processing device can determine the correlation coefficient between the local EEG signal j and each visual stimulus object k. The maximum value in the correlation coefficient is used to determine the visual stimulus corresponding to that local EEG signal as the initial visual stimulus. For example, if the maximum correlation coefficient is determined... If the value is the largest, then the visual stimulus object represented by k=5 is determined as the initial visual stimulus object corresponding to the local EEG signal j.

[0107] Step 6: Determine the characteristic values ​​corresponding to the local EEG signals based on the correlation coefficients between the local EEG signals and each visual stimulus object.

[0108] In practice, for each local EEG signal, the data processing device can sort the correlation coefficients between the local EEG signal and each visual stimulus object from largest to smallest, and take the top preset number (e.g., the first two) as the target correlation coefficients. Then, the data processing device can calculate the feature value corresponding to the local EEG signal based on the correlation coefficients and the target correlation coefficients. In one example, the feature value corresponding to local EEG signal j (which can be denoted as T) j The calculation formula for ) is as follows:

[0109]

[0110] Where N is the total number of visual stimuli. , These represent the correlation coefficients between local EEG signal j and N visual stimuli ( ). The maximum and second largest values ​​in the (k=1,2,…,N) are the two correlation coefficients that appear first after sorting from largest to smallest.

[0111] Step 7: When the initial visual stimulus object corresponding to each local EEG signal is the same and the feature value corresponding to each local EEG signal is greater than the preset threshold, the initial stimulus object is determined as the target visual stimulus object that the user is looking at.

[0112] In practice, the data processing device can compare the initial visual stimulus objects corresponding to each local EEG signal, and compare the feature values ​​of each local EEG signal with a preset threshold. If all initial visual stimuli are the same, and the feature values ​​of each local EEG signal are all greater than the preset threshold, the data processing device can determine the initial visual stimulus object as the target visual stimulus object for the user's gaze. The preset threshold can be set as needed, such as set to... .

[0113] In this example, based on the correlation coefficient between each local EEG signal and each visual stimulus object, the initial visual stimulus object and the corresponding feature value of each local EEG signal are identified respectively. Then, based on each initial visual stimulus object and the feature value, the target visual stimulus object is identified together, which can improve the recognition accuracy of operation commands.

[0114] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0115] Based on the same inventive concept, this application also provides a brain-computer interface-based display content control system for implementing the aforementioned brain-computer interface-based display content control method. The solution provided by this system is similar to the implementation scheme described in the above method. Therefore, the specific limitations of one or more embodiments of the brain-computer interface-based display content control system provided below can be found in the limitations of the brain-computer interface-based display content control method described above, and will not be repeated here.

[0116] In one embodiment, a brain-computer interface-based display content control system is provided, comprising: a terminal, an electroencephalogram (EEG) acquisition device, and a data processing device, wherein:

[0117] The terminal is used to display the content to be displayed in the content display area of ​​the display interface based on the current display frequency.

[0118] The EEG acquisition device is used to acquire the user's EEG signals while the user is reading the content displayed on the display interface.

[0119] The data processing device is used to determine the user's current reading state based on the acquired EEG signal and the current display frequency; determine a target display frequency based on the current reading state and a frequency update strategy; and send the target display frequency as the current display frequency to the terminal.

[0120] In one embodiment, the data processing device is specifically configured to: determine multiple filter frequency bands based on the current display frequency, and filter the acquired EEG signal based on the multiple filter frequency bands to obtain frequency band signals corresponding to each filter frequency band; use a typical correlation analysis algorithm to calculate the sub-correlation coefficient between each frequency band signal and the current display frequency, and determine the correlation coefficient between the EEG signal and the current display frequency based on each sub-correlation coefficient; and determine the user's current reading state based on the correlation coefficient.

[0121] In one embodiment, the data processing device is specifically configured to: determine the user's current reading state as a concentrated reading state when the correlation coefficient is greater than a preset threshold; and determine the user's current reading state as a dispersed reading state when the correlation coefficient is less than or equal to the preset threshold.

[0122] In one embodiment, the data processing device is specifically configured to: determine the current display frequency as the target display frequency when the user's current reading state is a concentrated reading state; and reduce the current display frequency based on a frequency reduction strategy when the user's current reading state is a dispersed reading state to obtain the target display frequency.

[0123] In one embodiment, the terminal is further configured to display visual stimuli corresponding to multiple operation controls in the control display area of ​​the display interface, with each visual stimuli flashing according to a different code. The EEG acquisition device is further configured to acquire the user's EEG signals while the user is looking at the control display area. The data processing device is further configured to parse and process the acquired EEG signals, identify the target visual stimuli being looked at by the user, and send the operation instructions corresponding to the target visual stimuli to the terminal. The terminal is further configured to execute the target operation based on the operation instructions sent by the data processing device.

[0124] Based on the same inventive concept, this application also provides a brain-computer interface-based display content control device for implementing the brain-computer interface-based display content control method described above. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more brain-computer interface-based display content control device embodiments provided below can be found in the limitations of the brain-computer interface-based display content control method described above, and will not be repeated here.

[0125] In one embodiment, such as Figure 6 As shown, a brain-computer interface-based display content control device 600 is provided, comprising: a first display module 601, a data acquisition module 602, a first determination module 603, and a second determination module 604, wherein:

[0126] The first display module 601 is used to display the content to be displayed in the content display area of ​​the display interface based on the current display frequency.

[0127] The acquisition module 602 is used to acquire the user's electroencephalogram (EEG) signals while the user is reading the content displayed on the display interface.

[0128] The first determining module 603 is used to determine the user's current reading state based on the collected EEG signals and the current display frequency.

[0129] The second determining module 604 is used to determine the target display frequency based on the current reading state and frequency update strategy, use the target display frequency as the current display frequency, and execute the step of displaying the content to be displayed in the content display area of ​​the display interface based on the current display frequency.

[0130] In one embodiment, the first determining module 603 is specifically configured to: determine multiple filtering frequency bands based on the current display frequency, and filter the acquired EEG signal based on the multiple filtering frequency bands to obtain frequency band signals corresponding to each filtering frequency band; use a typical correlation analysis algorithm to calculate the sub-correlation coefficient between each frequency band signal and the current display frequency, and determine the correlation coefficient between the EEG signal and the current display frequency based on each sub-correlation coefficient; and determine the user's current reading state based on the correlation coefficient.

[0131] In one embodiment, the first determining module 603 is specifically used to: determine the user's current reading state as a concentrated reading state when the correlation coefficient is greater than a preset threshold; and determine the user's current reading state as a dispersed reading state when the correlation coefficient is less than or equal to the preset threshold.

[0132] In one embodiment, the second determining module 604 is specifically configured to: determine the current display frequency as the target display frequency when the user's current reading state is a concentrated reading state; and reduce the current display frequency based on a frequency reduction strategy when the user's current reading state is a dispersed reading state to obtain the target display frequency.

[0133] In one embodiment, the device further includes a second display module, an identification module, and an execution module, wherein:

[0134] The second display module is used to display visual stimulus objects corresponding to multiple operation controls in the control display area of ​​the display interface, and each visual stimulus object flashes according to a different code.

[0135] The recognition module is used to collect the user's electroencephalogram (EEG) signals while the user is looking at the control display area, and to analyze and process the collected EEG signals to identify the target visual stimulus object that the user is looking at.

[0136] The execution module is used to perform the target operation based on the operation control corresponding to the target visual stimulus object.

[0137] The modules in the aforementioned brain-computer interface-based display content control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0138] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a brain-computer interface-based display content control method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0139] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0140] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0141] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0142] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0144] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0145] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0146] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for controlling display content based on a brain-computer interface, characterized in that, The method includes: Based on the current display frequency, display the content to be displayed in the content display area of ​​the display interface; While the user is reading the content displayed on the interface, the user's electroencephalogram (EEG) signals are collected; Based on the collected EEG signals and the current display frequency, the user's current reading state is determined; Based on the current reading state and frequency update strategy, a target display frequency is determined, the target display frequency is used as the current display frequency, and the step of displaying the content to be displayed in the content display area of ​​the display interface based on the current display frequency is executed; The step of determining the user's current reading state based on the collected EEG signals and the current display frequency includes: Based on the current display frequency, multiple filter frequency bands are determined, and the collected EEG signals are filtered based on the multiple filter frequency bands to obtain frequency band signals corresponding to each filter frequency band. The canonical correlation analysis algorithm is used to calculate the sub-correlation coefficient between each frequency band signal and the current display frequency, and the correlation coefficient between the EEG signal and the current display frequency is determined based on each sub-correlation coefficient; Based on the correlation coefficient, the user's current reading status is determined.

2. The method according to claim 1, characterized in that, Determining the user's current reading status based on the correlation coefficient includes: If the correlation coefficient is greater than a preset threshold, the user's current reading state is determined to be a concentrated reading state; If the correlation coefficient is less than or equal to a preset threshold, the user's current reading state is determined to be a scattered reading state.

3. The method according to any one of claims 1 to 2, characterized in that, The step of determining the target display frequency based on the current reading state and frequency update strategy includes: If the user's current reading state is focused reading, the current display frequency is determined as the target display frequency; If the user's current reading state is a scattered reading state, the current display frequency is reduced based on the frequency reduction strategy to obtain the target display frequency.

4. The method according to claim 1, characterized in that, The method further includes: In the control display area of ​​the display interface, visual stimulus objects corresponding to multiple operation controls are displayed, and each visual stimulus object flashes according to a different code; During the process of the user gazing at the control display area, the user's electroencephalogram (EEG) signals are collected, and the collected EEG signals are analyzed and processed to identify the target visual stimulus object that the user is gazing at. The target operation is executed based on the operation control corresponding to the target visual stimulus object.

5. A brain-computer interface-based display content control system, characterized in that, The system includes a terminal, an EEG acquisition device, and a data processing device, wherein: The terminal is used to display content to be displayed in the content display area of ​​the display interface based on the current display frequency; The EEG acquisition device is used to acquire the user's EEG signals while the user is reading the content displayed on the display interface; The data processing device is configured to: determine multiple filtering frequency bands based on the current display frequency; filter the acquired EEG signal based on the multiple filtering frequency bands to obtain frequency band signals corresponding to each filtering frequency band; calculate the sub-correlation coefficient between each frequency band signal and the current display frequency using a canonical correlation analysis algorithm; determine the correlation coefficient between the EEG signal and the current display frequency based on each sub-correlation coefficient; determine the user's current reading state based on the correlation coefficient; determine the target display frequency based on the current reading state and the frequency update strategy; and send the target display frequency as the current display frequency to the terminal.

6. A display content control device based on a brain-computer interface, characterized in that, The device includes: The display module is used to display the content to be displayed in the content display area of ​​the display interface based on the current display frequency; The acquisition module is used to acquire the user's electroencephalogram (EEG) signals while the user is reading the content displayed on the display interface; The first determining module is configured to: determine multiple filtering frequency bands based on the current display frequency; filter the acquired EEG signal based on the multiple filtering frequency bands to obtain frequency band signals corresponding to each filtering frequency band; calculate the sub-correlation coefficient between each frequency band signal and the current display frequency using a canonical correlation analysis algorithm; determine the correlation coefficient between the EEG signal and the current display frequency based on each sub-correlation coefficient; and determine the user's current reading state based on the correlation coefficient. The second determining module is used to determine the target display frequency based on the current reading state and frequency update strategy, use the target display frequency as the current display frequency, and execute the step of displaying the content to be displayed in the content display area of ​​the display interface based on the current display frequency.

7. The apparatus according to claim 6, characterized in that, The second determining module is used to determine the current display frequency as the target display frequency when the user's current reading state is a concentrated reading state; If the user's current reading state is a scattered reading state, the current display frequency is reduced based on the frequency reduction strategy to obtain the target display frequency.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images