Multi-interface Fusion Dynamic Brain-Computer Interaction Method and Terminal Based on Frequency-Phase Information Multiplexing

Abstract

The present invention relates to the technical field of brain-computer interfaces, and discloses a multi-interface fusion dynamic brain-computer interaction method and terminal based on frequency-phase information multiplexing. Based on the user's offline electroencephalogram (EEG) experiment, the maximum number U of EEG stimulations for the user to achieve the optimal EEG stimulation classification effect, the frequency-phase information of the U-classified EEG stimulations, and the first correspondence between the U-classified EEG stimulations and the EEG signal characteristics are obtained. The interactive interface after multi-interface fusion is dynamically partitioned to obtain Z sub-regions and the brain control instructions included in each sub-region. The frequency-phase information of the U-classified EEG stimulations is assigned to each brain control instruction. Based on the EEG signal characteristics induced by the flashing rule of the EEG stimulation corresponding to a certain brain control instruction when the user gazes at it, the EEG stimulation classification is obtained based on the first correspondence. The sub-region is determined based on the user's line-of-sight landing coordinates. Combining the EEG stimulation classification and the determined sub-region, a brain control instruction is output. The present invention can improve the efficiency of brain-computer interaction operations.

Description

Technical Field

[0001] The present invention relates to the technical field of brain-computer interfaces, and in particular to a multi-interface fusion dynamic brain-computer interaction method and terminal based on frequency-phase information multiplexing. Background Art

[0002] Brain-computer interface (BCI) technology is an interdisciplinary technology involving neuroscience, signal processing, pattern recognition, etc. It can directly convert the information sent by the brain into commands that can drive external devices, and replace human limbs or language to achieve communication between humans and the outside world and control of the external environment. Currently, BCI technology is widely used in the field of intelligent control and has been able to achieve the control of various devices such as robotic arms, wheelchairs, exoskeletons, and drones.

[0003] Nowadays, with the expansion of the control dimension of the controlled object and the installation of combined devices, the number of brain control instruction classifications on the interaction interface is increasing continuously, and the demand for the types of stimulus frequency-phase encoding combinations is increasing. However, the current number of available stimulus encoding combinations is still very limited. Especially in the case of high-frequency stimuli with high user comfort but low classification accuracy, it is difficult to meet the demand for large instruction classification stimulus encoding of users, and the frequency-phase information with small differences between stimulus blocks will reduce the classification accuracy of users. For example, in the case of online control of drones, users need to simultaneously consider the environmental information of the first-person view video transmission, and the user's electroencephalogram (EEG) load is large. If large instruction classification is always carried out, the user's fatigue level will increase and the EEG load will be overloaded, resulting in low control efficiency and poor control accuracy of the user. At the same time, as the application of the brain-computer interaction system gradually moves from indoors to outdoors and from a single laboratory scenario to a complex outdoor scenario, the environmental information that users need to pay attention to increases, and the control process becomes gradually complicated. How to reduce the user's EEG load, limit the user's misoperations, and improve the brain-computer control efficiency during the control process has become an urgent problem to be solved.

[0004] In complex time-varying scenarios, as the scene information is continuously updated, at certain moments, the user's manipulation requirements only cover actions in specific dimensions, only involving the classification of a small number of brain control commands. The user interface only needs to present a specific number of brain control stimuli. However, currently, traditional static interaction interfaces cannot present different stimuli at different times. All stimuli are presented at the beginning of the task, and the number, position, and frequency-phase information of the stimuli must remain unchanged until the end of the task. This method will result in a high redundancy of brain control commands, a large brain electrical load on the user, an increase in the user's fatigue level in complex time-varying scenarios, and the redundant and irrelevant commands will increase the user's misoperations, posing great risks to both the user and the controlled object. In addition, existing brain-computer interaction interfaces are no longer satisfied with a single display device. Multiple display devices are often required to present the multi-channel feedback video information of the controlled object and the combined carrier in different regions, and display the brain control stimuli corresponding to specific operations in specific regions. At the same time, along with the update of the scene information during the interaction process, the parameter information such as the position, size, and mapped command name of the stimuli in different regions also needs to be updated in a time-varying manner. Moreover, during the process of inducing brain electrical stimuli, the synchronization of the brain electrical tags sent by the interface needs to be maintained at all times. During the online decoding process of brain electricity, the timestamps of all regions of all devices need to be consistent. Therefore, how to synchronize and unify the clocks of multiple distributed devices has also become an important issue for the display device of the brain-computer interaction interface to develop from single to extended.

[0005] In view of this, it is urgent to develop a dynamic brain-computer interaction interface that can solve the problems of high redundancy of commands, poor timing flexibility, large coding difficulty, small environmental information volume, and single device display in existing brain-computer interaction interfaces. Summary of the Invention

[0006] The purpose of the present invention is to provide a multi-interface fusion dynamic brain-computer interaction method and terminal based on frequency-phase information multiplexing, which can filter out irrelevant control commands according to the actual situation, fuse multiple interfaces into an interactive interface and perform dynamic partitioning, and perform frequency-phase information multiplexing allocation on the brain control commands in the sub-regions, which can greatly improve the interface interaction speed, reduce the user's task load, limit the user's misoperations, and can shorten the output command time and improve the brain-computer interaction operation efficiency.

[0007] To achieve the above purpose, the present invention provides the following technical solutions:

[0008] In the first aspect, the present invention provides a multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing. First, configure the user's offline electroencephalogram experiment to obtain the maximum number U of brain electrical stimulus classifications for the user to achieve the optimal brain electrical stimulus classification effect, the frequency-phase information of the U-classified brain electrical stimuli, and the first correspondence between the U-classified brain electrical stimuli and the brain electrical signal characteristics. The dynamic brain-computer interaction method includes:

[0009] S10. Integrate multiple interfaces into one interactive interface, align the clock frequencies of the multiple interfaces, and unify the EEG label timestamps of the interactive interface;

[0010] S11. Merge the resolutions of the multiple interfaces to obtain the resolution of the interactive interface;

[0011] S12. When the brain control instructions change over time, configure the N brain control instructions required to be presented at a certain moment t, assign EEG stimuli to the N brain control instructions, and obtain the second correspondence between the brain control instructions and the EEG stimuli;

[0012] S13. Dynamically partition the interactive interface based on the resolution of the interactive interface and the maximum number U of EEG stimulus classifications, obtain Z sub-regions and the brain control instructions included in each sub-region;

[0013] S14. Assign the frequency-phase information of the U-class EEG stimuli to each brain control instruction in each sub-region. At this time, the EEG stimuli corresponding to each brain control instruction in the sub-region flash and appear according to the assigned frequency-phase information, and the flashing rule of the EEG stimuli is obtained;

[0014] S15. At moment t, when the user gazes at a certain brain control instruction, the flashing rule of the EEG stimulus corresponding to the brain control instruction induces EEG signal characteristics. Based on the EEG signal characteristics and the first correspondence, the corresponding EEG stimulus classification is obtained;

[0015] S16. Obtain the pixel coordinates of the gaze landing point on the interactive interface when the user gazes at the brain control instruction at moment t, and determine the sub-region where the gaze landing point is located;

[0016] S17. Based on the EEG stimulus classification obtained in S15 and the sub-region determined in S16, determine the brain control instruction required by the user and output it to achieve dynamic brain-computer interaction.

[0017] As a possible implementation, S14 is specifically: in ascending order of the brain control instruction coordinates in each sub-region, assign the frequency-phase information of the U-class EEG stimuli to the brain control instructions in each sub-region in turn, and the frequency-phase information corresponding to the brain control instructions in the same sub-region does not repeat.

[0018] As a possible implementation, the following method is used to obtain the resolution of the interactive interface:

[0019] When the multiple interfaces are arranged in a single column, the resolution of the interactive interface is [X, Y], X = R i , where, R i is the minimum number of row pixels in the multiple interfaces, C k is the number of column pixels of the kth interface, M is the total number of interfaces, X is the total number of row pixels of the interactive interface, and Y is the total number of column pixels of the interactive interface;

[0020] When multiple interfaces are arranged in a single row, the resolution of the interactive interface is [X, Y], Y = C i ; where R k is the number of row pixels of the kth interface, and C i is the minimum number of column pixels in the multiple interfaces, M is the total number of interfaces, X is the total number of row pixels of the interactive interface, and Y is the total number of column pixels of the interactive interface.

[0021] As a possible implementation, S13 includes: a dynamic partition where the positions of the brain - controlled instructions are not fixed, specifically:

[0022] S130. Round up the quotient obtained by dividing the number N of brain - controlled instructions by the maximum number U of electroencephalogram stimulation classifications to obtain Z sub - regions;

[0023] S131. Allocate the N brain - controlled instructions to the Z sub - regions in sequence. The number of brain - controlled instructions in the first Z - 1 sub - regions is U each, and the number of brain - controlled instructions in the Zth sub - region is N - U×(Z - 1).

[0024] As a possible implementation, S131 includes the following sub - steps:

[0025] S1310. Obtain the number of pixel points covered by each sub - region based on the resolution of the interactive interface and the number Z of sub - regions: Z i represents the number of pixel points covered by the ith sub - region, i = 1, 2,..., Z;

[0026] S1311. If the interactive interface is partitioned by row pixels, the row - pixel range of each sub - region is and the column - pixel range is (0, Y];

[0027] If the interactive interface is partitioned by column pixels, the row - pixel range of each sub - region is (0, X], and the column - pixel range is where X is the total number of row pixels of the interactive interface, Y is the total number of column pixels of the interactive interface, Z is the number of sub - regions, i represents the ith sub - region, i = 1, 2,..., Z;

[0028] S1312. Determine the central - position coordinates of each brain - controlled instruction in each sub - region, specifically:

[0029] If the interactive interface is partitioned by row pixels, the central - position coordinates of the vth brain - controlled instruction in the ith sub - region i v are:

[0030]

[0031] If the interactive interface is partitioned by column pixels, the central position coordinates of the v-th brain control instruction in the i-th sub-region are: v as follows:

[0032]

[0033] where i represents the i-th sub-region, i is less than or equal to Z, and i v represents the v-th brain control instruction in the i-th sub-region, S i represents the number of brain control instructions in the i-th sub-region. The brain control instructions are square, l represents the side length of the brain control instruction, and d represents the spacing between brain control instructions.

[0034] As a possible implementation, S13 includes: dynamic partitioning with fixed brain control instruction positions. At this time, the central position coordinates of each brain control instruction are known, specifically:

[0035] S133. Read the central position coordinates of each brain control instruction and sort the N brain control instructions according to their central position coordinates;

[0036] S134. For the i-th sub-region, judge the size relationship between the number of brain control instructions to be partitioned and U; when the number of brain control instructions to be partitioned is greater than U, execute S135; when the number of brain control instructions to be partitioned is less than or equal to U, execute S136;

[0037] S135. Divide the first U brain control instructions among the brain control instructions to be partitioned into the same sub-region, and judge whether there is an overlap between the pixel range of this sub-region and the pixel range of the (U + 1)-th brain control instruction among the brain control instructions to be partitioned. If not, the number of brain control instructions in this sub-region is U, let i = i + 1 and continue to execute S134; otherwise, let U = U - 1 and re-execute S135;

[0038] S136. Divide the brain control instructions to be partitioned into the same sub-region, complete the dynamic partitioning, and obtain Z sub-regions and the brain control instructions included in each sub-region.

[0039] As a possible implementation, the pixel range of the sub-region in S135 is obtained based on the central coordinates of the brain control instructions partitioned into this sub-region, specifically:

[0040] S1350. Obtain the minimum value x min and the maximum value x max of the row pixel coordinates, and the minimum value y min and the maximum value y max of the column pixel coordinates among the central position coordinates of the brain control instructions partitioned into this sub-region;

[0041] S1351. The row pixel range of this sub-region is [x min - l, xmax +l), and the column pixel range is [y min -l, y max +l); wherein, the brain control instruction is a square, and l represents the side length of the brain control instruction.

[0042] As a possible implementation, S133 is specifically: reading the central position coordinates of each brain control instruction, arranging the N brain control instructions in ascending order of the column pixel coordinates in their central position coordinates, and when the column pixel coordinates are the same, arranging them in ascending order of the horizontal pixel coordinates.

[0043] In a second aspect, the present invention provides a terminal, including a processor and a communication interface coupled to the processor, where the processor is used to run a computer program or instruction to implement the multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing provided in the first aspect.

[0044] Compared with the prior art, the beneficial effects produced by the present invention are as follows:

[0045] 1. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing provided by the present invention, through the mode setting of independent decoupling and transparent penetration, solves the problems of limited vision of the existing brain-computer interaction interface and limited presentation of environmental information volume. At the same time, this method aligns the EEG tag timestamps of multiple interfaces, solves the problem of inconsistent EEG stimulation paradigm tags across interfaces, and maximally drives the performance of the graphics card, reduces the device display latency, improves the refresh speed of the EEG stimulation flashing frames, and realizes good induction of EEG signals under the cross-interface stimulation paradigm.

[0046] 2. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing provided by the present invention is based on the optimal classification results obtained from the user's offline experiment, which guarantees the individual differences of the subjects. Regardless of whether the EEG classification ability of the subjects is superior, it can optimize the user's behavior to the greatest extent and reduce the user's misoperations. At the same time, this dynamic partitioning method can adapt to various complex scenarios, broadens the application scenarios of the brain-computer interaction interface, can promote the brain control system from indoor to outdoor, and improves the stability and robustness of the brain-computer interaction system.

[0047] 3. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing provided by the present invention adopts the frequency-phase information multiplexing of visual evoked stimuli, can be applied to a variety of visual stimulation paradigms, and is applicable in both low-frequency and high-frequency cases. It solves the problem of limited number of stimulus coding combinations in the classification of large EEG instructions, especially in the case of high-frequency stimuli with higher comfort. When the regional information can be accurately divided, this method can greatly reduce the number of EEG stimulation classifications and guarantee the optimal EEG stimulation classification results. Description of the Drawings

[0048] The accompanying drawings described herein are used to provide a further understanding of the present invention and form a part of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0049] Figure 1 is the traditional brain-controlled drone interface in the embodiment of the present invention;

[0050] Figure 2 is the flowchart of the multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing in the embodiment of the present invention;

[0051] Figure 3 is the schematic diagram of the interactive interface divided by row pixels in the case of dynamic partitioning with unfixed brain control instruction positions in the embodiment of the present invention;

[0052] Figure 4 is the schematic diagram of the interactive interface divided by column pixels in the case of dynamic partitioning with unfixed brain control instruction positions in the embodiment of the present invention;

[0053] Figure 5 is the schematic diagram of dynamic partitioning with fixed brain control instruction positions in the embodiment of the present invention;

[0054] Figure 6 is the schematic diagram of allocating frequency-phase information to the brain control instructions in each sub-region when the interactive interface is divided by column pixels in the case of dynamic partitioning with unfixed brain control instruction positions in the embodiment of the present invention;

[0055] Figure 7 is the schematic diagram of allocating frequency-phase information to the brain control instructions in each sub-region in the case of dynamic partitioning with fixed brain control instruction positions in the embodiment of the present invention;

[0056] Figure 8 is the schematic diagram of the need to fix the position of the sub-region according to the preset in the embodiment of the present invention;

[0057] Figure 9 is the schematic diagram of fusing two interfaces into an interactive interface in the embodiment of the present invention;

[0058] Figure 10 is the result of dynamic partitioning and frequency-phase information allocation with unfixed brain control instruction positions in the embodiment of the present invention;

[0059] Figure 11 is the result of dynamic partitioning and frequency-phase information allocation with fixed brain control instruction positions in the embodiment of the present invention. Detailed implementation manners

[0060] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and roles. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and do not limit their sequence. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and the terms "first" and "second" do not necessarily mean different.

[0061] It should be noted that in the present invention, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0062] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. The following at least one (item) or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b or c can represent: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b and c can be single or multiple.

[0063] Currently, with the continuous expansion of the application scenarios of BCI systems and the continuous increase in the control dimensions, traditional brain-computer interaction interfaces have gradually been unable to meet the application requirements of current BCI systems. Taking a brain-controlled drone as an example, such as Figure 1As shown in the traditional brain-controlled drone interface, in the traditional static brain-computer interaction interface, the video stream of the first-person view of the drone is continuously presented in the center of the interface, and steady-state visual evoked potential (SSVEP) stimulus flashing blocks are distributed around it. The 12-instruction classification brain electrical stimulation is mapped to 12 flight actions of the drone. The stimulus blocks correspond to 12 different frequencies and specific phases according to joint frequency-phase modulation (JFPM). During the process of the drone completing the task, the stimulus blocks need to be presented throughout. The user has been performing the 12-instruction classification task of brain electricity during the experiment. The traditional static brain-computer interaction interface will result in a relatively high redundancy of brain control instructions, a relatively large brain electrical load of the user, an increase in the fatigue level of the user in complex time-varying scenarios, and the redundant irrelevant instructions will increase the user's misoperation, posing greater risks to both the user and the controlled object.

[0064] The present invention proposes a multi-interface fusion dynamic brain-computer interaction method and terminal based on frequency-phase information multiplexing, which can filter out irrelevant control instructions according to the actual situation, fuse multiple interfaces into an interactive interface and perform dynamic partitioning, and perform frequency-phase information multiplexing allocation on the brain control instructions in the sub-regions, which can greatly improve the interface interaction speed, reduce the user's task load, limit the user's misoperation, and can shorten the instruction output time and improve the brain-computer interaction operation efficiency.

[0065] In the first aspect, the present invention provides a multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing. First, configure the user's offline brain electrical experiment to obtain the maximum number of brain electrical stimulation classifications U that the user achieves the optimal brain electrical stimulation classification effect, the frequency-phase information of the U-class brain electrical stimulation, and the first correspondence between the U-class brain electrical stimulation and the brain electrical signal characteristics.

[0066] As an example, assume that the number of brain electrical stimulation classifications is U s and the number of brain electrical stimulation classifications is U l when the optimal brain electrical classification results obtained from the experiment are the same, but U s is less than U l , then the maximum number of brain electrical stimulation classifications U is equal to U l , that is, the maximum value of the two. The U-class brain electrical stimulations are respectively denoted as: U1 class, U2 class,..., U u class, and the frequency-phase information of each class of brain electrical stimulation is denoted as: (Fre u , Pha u ), u = 1, 2,..., U, and the first correspondence is recorded as shown in Table 1:

[0067] Table 1 The first correspondence between U-classified brain electrical stimulation and brain electrical signal characteristics

[0068] EEG Stimulation Classification EEG Signal Features <![CDATA[U1]]> Feature 1 <![CDATA[U2]]> Feature 2 <![CDATA[U3]]> Feature 3 <![CDATA[U4]]> Feature 4 …… ……

[0069] Based on the optimal classification results obtained from the user's offline experiment, the present invention ensures the individual differences of the subjects. Regardless of whether the subjects' brain electrical classification ability is superior or not, it can optimize the user's behavior to the greatest extent and reduce the user's misoperation. At the same time, this dynamic partitioning method can adapt to various complex scenarios, broaden the application scenarios of the brain-computer interface, and promote the brain control system to move from indoors to outdoors, improving the stability and robustness of the brain-computer interaction system.

[0070] See Figure 2 , the dynamic brain-computer interaction method includes:

[0071] S10. Integrate multiple interfaces into one interactive interface, align the clock frequencies of the multiple interfaces, and unify the EEG label timestamps of the interactive interface;

[0072] As an example, assume that the total number of interface display devices is M. Merge the interface canvases across multiple display devices and set them to the transparent penetration mode, so that the multiple interfaces are integrated into one interactive interface, which can be independently superimposed on any static or dynamic background; Connect the M display devices to the same graphics card, and align the clock frequencies of the multiple interfaces through the graphics card surround setting to keep the interfaces refreshed synchronously and unify the EEG label timestamps of the interactive interface.

[0073] The present invention aligns the EEG label timestamps of the multiple interfaces, solves the problem of inconsistent labels of the cross-interface brain electrical stimulation paradigm, and drives the performance of the graphics card to the greatest extent, reduces the device display latency, improves the refresh speed of the EEG stimulation flicker frames, and realizes the good induction of the EEG signals under the cross-interface stimulation paradigm.

[0074] S11. Merge the resolutions of the multiple interfaces to obtain the resolution of the interactive interface;

[0075] As a possible implementation method, the following method is used to obtain the resolution of the interactive interface:

[0076] When the multiple interfaces are arranged in a single column, the resolution of the interactive interface is [X, Y], X = R i , where R i is the minimum number of row pixels in the multiple interfaces, C k is the number of column pixels of the kth interface, M is the total number of interfaces, X is the total number of row pixels of the interactive interface, and Y is the total number of column pixels of the interactive interface;

[0077] When the multiple interfaces are arranged in a single row, the resolution of the interactive interface is [X, Y], Y = C i; where R k is the number of row pixels of the k-th interface, and C i is the minimum number of column pixels in the multi-interface, M is the total number of interfaces, X is the total number of row pixels of the interactive interface, and Y is the total number of column pixels of the interactive interface.

[0078] S12. When the brain control instructions change over time, configure the N brain control instructions required to be presented at a certain moment t, allocate electroencephalogram (EEG) stimulation to the N brain control instructions, and obtain the second correspondence between the brain control instructions and the EEG stimulation;

[0079] As an example, assume that the number of brain control instructions required to be presented at time t is 10. Name these 10 brain control instructions as A to J, allocate EEG stimulation to the 10 brain control instructions, and the second correspondence between the brain control instructions and the EEG stimulation obtained is shown in Table 2:

[0080] Table 2 The second correspondence between brain control instructions and EEG stimulation

[0081] Brain-Controlled Command EEG Stimulation A Stimulation 1 B Stimulation 2 C Stimulation 3 D Stimulation 4 …… …… J Stimulation 10

[0082] S13. Dynamically partition the interactive interface based on the resolution of the interactive interface and the maximum number U of EEG stimulation classifications, and obtain Z sub-regions and the brain control instructions included in each sub-region;

[0083] As a possible implementation method, the dynamic partitioning includes: dynamic partitioning with non-fixed positions of brain control instructions. In this case, the display positions of brain control instructions on the interactive interface can be freely arranged. Specifically:

[0084] S130. Round up the quotient obtained by dividing the number N of brain control instructions by the maximum number U of EEG stimulation classifications to obtain Z sub-regions; that is Z = 1, 2, …

[0085] S131. Allocate the N brain control instructions to the Z sub-regions in sequence. The number of brain control instructions in the first Z - 1 sub-regions is U, and the number of brain control instructions in the Z-th sub-region is N - U×(Z - 1).

[0086] As a possible implementation method, S131 includes the following sub-steps:

[0087] S1310. Based on the resolution of the interactive interface and the number Z of sub-regions, obtain the number of pixel points covered by each sub-region: Z i represents the number of pixel points covered by the i-th sub-region, i = 1, 2, …, Z, X is the total number of row pixels of the interactive interface, and Y is the total number of column pixels of the interactive interface;

[0088] S1311. If the interactive interface is partitioned by row pixels, the row pixel range of each sub-region is The column pixel range is (0, Y]; as can be seen from S1310, the number of pixel points covered by each sub-region is equal. Therefore, when partitioning by row pixels, the column pixel range of each sub-region is (0, Y], while the row pixel ranges are different.

[0089] As an example of partitioning by row pixels, refer to Figure 3 , assuming that the interactive interface is divided into Z sub-regions from left to right, the row pixel range of the first sub-region is The row pixel range of the second sub-region is And so on, the row pixel range of the i-th sub-region can be obtained as

[0090] If the interactive interface is partitioned by column pixels, the row pixel range of each sub-region is (0, X], and the column pixel range is where X is the row pixel of the interactive interface, Y is the column pixel of the interactive interface, Z is the number of sub-regions, i represents the i-th sub-region, i = 1, 2,..., Z;

[0091] S1312. Determine the central position coordinates of each brain control instruction in each sub-region, specifically:

[0092] If the interactive interface is partitioned by row pixels, the v-th brain control instruction i in the i-th sub-region v has the central position coordinates:

[0093]

[0094] If the interactive interface is partitioned by column pixels, the v-th brain control instruction i in the i-th sub-region v has the central position coordinates:

[0095]

[0096] In the formula, i represents the i-th sub-region, i is less than or equal to Z, i v represents the v-th brain control instruction in the i-th sub-region, S i represents the number of brain control instructions in the i-th sub-region. The brain control instructions are square, l represents the side length of the brain control instruction, and d represents the spacing between the brain control instructions.

[0097] As an example of partitioning the interactive interface by column pixels, refer to Figure 4 , assuming U is 4, and 10 brain control instructions A - J are sequentially assigned to In the sub-regions, the number of brain control instructions in the first two sub-regions is 4 each, and the number of brain control instructions in the third sub-region is 2. Generally, to maintain good classification results, the size and spacing of the brain control instructions are kept consistent, and the brain control instructions are mostly square. Let l represent the side length of the brain control instruction and d represent the spacing between the brain control instructions. To maximize the classification accuracy of the user, when the positions of the brain control instructions are not fixed, the brain control instructions are placed in the central vision of each sub-region as much as possible. The number of column pixels in each sub-region is So the distance from the midpoint of the column pixels of each sub-region to the upper and lower edges of the sub-region is the same, both being Therefore, the column pixel coordinate of the center point of the v-th brain control instruction in the i-th sub-region can be obtained as:

[0098] The number of row pixels in each sub-region is X, and the midpoint of the row pixels is To ensure that the brain control instructions are placed in the central vision of each sub-region as much as possible, starting from the midpoint of the row pixels of the region stimulations are arranged from the midline of the row pixels of the region to both ends. Then, taking Figure 4 region 1 as an example, the row pixel coordinate of the B instruction closest to the midline is The row pixel coordinate of the C instruction is Secondly, the row pixel coordinate of the A instruction is The row pixel coordinate of the D instruction is Therefore, the row pixel coordinate of the center point of the v-th brain control instruction in the i-th sub-region can be obtained as:

[0099] See Figure 4 , and by solving, the central position coordinate of the 3rd brain control instruction in the 2nd sub-region can be obtained as Since the brain control instructions are distributed in sequence, the 3rd brain control instruction to be placed in the 2nd sub-region, that is, the 7th brain control instruction, is G. Therefore, the brain control instruction G can be placed at its central position coordinate until the dynamic partitioning of all instructions is completed.

[0100] As a possible implementation, the dynamic partitioning includes: the dynamic partitioning with fixed positions of the brain control instructions, that is, the brain control instructions need to be fixed at a certain position on the interface according to actual needs. At this time, the central position coordinate of each brain control instruction is known, specifically:

[0101] S133. Read the central position coordinates of each brain control instruction, and sort the N brain control instructions according to their central position coordinates;

[0102] As a possible implementation, sort the N brain control instructions according to their central position coordinates. Specifically: read the central position coordinates of each brain control instruction, and sort the N brain control instructions in ascending order of the column pixel coordinates in their central position coordinates. When the column pixel coordinates are the same, sort them in ascending order of the row pixel coordinates.

[0103] As an example, see Figure 5 , given the position coordinates of 10 brain control instructions A - J, sorted in ascending order of the column coordinates in their central position coordinates as ABECFIGDJH. Among them, the column coordinates of the position coordinates of G and D are the same, so the two are sorted in ascending order of the row pixel coordinates, that is, G is before D.

[0104] S134. For the i-th sub-region, judge the number of brain control instructions to be partitioned and the size of U; when the number of brain control instructions to be partitioned is greater than U, execute S135; when the number of brain control instructions to be partitioned is less than or equal to U, execute S136;

[0105] S135. Partition the first U brain control instructions among the brain control instructions to be partitioned into the same sub-region, and judge whether there is an overlap between the pixel range of this sub-region and the pixel range of the (U + 1)-th brain control instruction among the brain control instructions to be partitioned. If not, the number of brain control instructions in this sub-region is U, let i = i + 1 and continue to execute S134; otherwise, let U = U - 1 and re-execute S135;

[0106] As a possible implementation, the pixel range of the sub-region is obtained based on the central coordinates of the brain control instructions partitioned into this sub-region. Specifically:

[0107] S1350. Obtain the minimum value x min and the maximum value x max of the row pixel coordinates in the central position coordinates of the brain control instructions partitioned into this sub-region, as well as the minimum value y min and the maximum value y max of the column pixel coordinates;

[0108] S1351. The row pixel range of this sub-region is [x min - l, x max + l), and the column pixel range is [y min - l, y max + l); where the brain control instruction is a square, and l represents the side length of the brain control instruction.

[0109] S136. Partition the brain control instructions to be partitioned into the same sub-region, complete the dynamic partition, and obtain Z sub-regions and the brain control instructions included in each sub-region.

[0110] As an example of dynamic partitioning with a fixed position of brain - controlled instructions, assume that the value of U is 4. At the first partition, the brain - controlled instructions to be partitioned are ABECFIGDJH, and the number is 10, which is greater than 4. Therefore, the first 4 brain - controlled instructions ABEC are divided into the same sub - region, and it is judged whether the pixel range of this sub - region overlaps with the pixel range of the 5th brain - controlled instruction, that is, F, among the brain - controlled instructions to be partitioned. As can be seen from Figure 5 it, among the central coordinates of the four brain - controlled instructions ABEC, the brain - controlled instruction with the smallest row pixel coordinate value is A, and its row pixel coordinate value is The brain - controlled instruction with the largest row pixel coordinate value is C, and its row pixel coordinate value is The brain - controlled instruction with the smallest column pixel coordinate value is A, and its column pixel coordinate value is The brain - controlled instruction with the largest column pixel coordinate value is C, and its column pixel coordinate value is Therefore, the row pixel range of this sub - region is The column pixel range is It should be explained that when obtaining the pixel information of the fixation area using an eye tracker or computer vision technology, there is a certain accuracy error. Therefore, when dividing the region, a error margin is maintained. If the value after - l is negative, it starts from zero. The central position coordinates of the brain - controlled instruction F are So its row pixel range is The column pixel range is Judge whether the row pixel range of this sub - region The column pixel range overlaps with the row pixel range of the brain - controlled instruction F The column pixel range If there is no overlap, the number of brain - controlled instructions in this sub - region is 4, that is: ABEC. Then the number of brain - controlled instructions to be partitioned is updated to 6: FIGDJH. The first 4 of these 6 brain - controlled instructions are divided into the next sub - region, and the above operation is repeated. If there is an overlap, the number of brain - controlled instructions in this sub - region is updated to 3, that is: ABE. It is judged whether the pixel range of the sub - region where these three brain - controlled instructions are located overlaps with the pixel range of the 4th brain - controlled instruction, that is, C. If not, the number of brain - controlled instructions in this sub - region is 3, that is: ABE. Then the number of brain - controlled instructions to be partitioned is updated to 7: CFIGDJH. If there is an overlap, the number of brain - controlled instructions in this sub - region is updated to 2, that is: AB. Continue to judge whether the pixel range of the sub - region where these two brain - controlled instructions are located overlaps with the pixel range of the 3rd brain - controlled instruction, that is, E.

[0111] When the number of brain - controlled instructions to be partitioned is less than 4, for example, the number of brain - controlled instructions to be partitioned is 2, that is: JH, see Figure 5, then JH is divided into the same sub-region, and the dynamic partitioning is completed. In this example, a total of 5 sub-regions are obtained. The number of brain control instructions in the first sub-region is 4: ABEC, the number of brain control instructions in the second sub-region is 1: F, the number of brain control instructions in the third sub-region is 1: I, the number of brain control instructions in the fourth sub-region is 2: GD, and the number of brain control instructions in the fifth sub-region is 2: JH.

[0112] S14. Assign the frequency-phase information of the U-class EEG stimulation to each brain control instruction in each sub-region. At this time, the EEG stimulation corresponding to each brain control instruction in the sub-region flashes according to the assigned frequency-phase information, and the flashing rule of the EEG stimulation is obtained;

[0113] As a possible implementation, in ascending order of the coordinates of the brain control instructions in each sub-region, the frequency-phase information of the U-class EEG stimulation is sequentially assigned to the brain control instructions in each sub-region, and the frequency-phase information corresponding to the brain control instructions in the same sub-region does not repeat.

[0114] As an example, assume that the number of Us is 4, and the frequency-phase information of this U-class EEG stimulation recorded during the offline EEG experiment is: U1: (Fre1, Pha1), U2: (Fre2, Pha2), U3: (Fre3, Pha3), U4: (Fre4, Pha4). Since Figure 4 What is shown is the result of partitioning by column pixels in the case of dynamic partitioning where the positions of the brain control instructions are not fixed. Therefore, here, the result of this partitioning is continued to show how to assign the frequency-phase information of the U-class EEG stimulation to each brain control instruction in each sub-region in the case of dynamic partitioning where the positions of the brain control instructions are not fixed. For each sub-region, in ascending order of the coordinates of the brain control instructions, the frequency-phase information of the U-class EEG stimulation is sequentially assigned to the brain control instructions. In the partitioning result by column pixels, the column pixels of the central position coordinates of the brain control instructions in each sub-region are the same. Therefore, it is assigned in ascending order of the row pixels of the central position coordinates. For the assignment result, see Figure 6, the frequency-phase information obtained for brain control instructions A to D in the first sub-region is successively: (Fre1, Pha1), (Fre2, Pha2), (Fre3, Pha3), (Fre4, Pha4), and the frequency-phase information obtained for brain control instructions E to H in the second sub-region is successively: (Fre1, Pha1), (Fre2, Pha2), (Fre3, Pha3), (Fre4, Pha4). The frequency-phase information obtained for brain control instructions I to J in the third sub-region is successively: (Fre1, Pha1), (Fre2, Pha2). Referring to Table 2, it can be known that the electroencephalogram (EEG) stimulations corresponding to each brain control instruction are as follows. For example, the EEG stimulations corresponding to brain control instructions A to D in the first sub-region are successively Stimulation 1, Stimulation 2, Stimulation 3, Stimulation 4, and the EEG stimulations corresponding to brain control instructions E to H in the second sub-region are successively Stimulation 5, Stimulation 6, Stimulation 7, Stimulation 8. Since the frequency-phase information assigned to brain control instructions A and E is the same, after the assignment is completed, the flashing rules of Stimulation 1 and Stimulation 5 are also the same.

[0115] As an example, since Figure 5 the displayed partitioning result is for the case of dynamic partitioning with fixed brain control instruction positions, the partitioning result will continue to be used to show how to assign the frequency-phase information of the U-class EEG stimulation to each brain control instruction in each sub-region in the case of dynamic partitioning with fixed brain control instruction positions. Since the brain control instructions in the dynamic partitioning with fixed brain control instruction positions have been arranged in ascending order of the central position coordinates of the brain control instructions before partitioning, for the brain control instructions in each sub-region, the frequency-phase information can be assigned according to their arrangement positions in sequence. For the assignment result, refer to Figure 7 , the frequency-phase information obtained for brain control instructions A, B, E, C in the first sub-region is successively: (Fre1, Pha1), (Fre2, Pha2), (Fre3, Pha3), (Fre4, Pha4), the frequency-phase information obtained for brain control instruction F in the second sub-region is: (Fre1, Pha1), the frequency-phase information obtained for brain control instruction I in the third sub-region is: (Fre1, Pha1), the frequency-phase information obtained for brain control instructions G, D in the fourth sub-region is successively: (Fre1, Pha1), (Fre2, Pha2), and the frequency-phase information obtained for brain control instructions J, H in the fifth sub-region is successively: (Fre1, Pha1), (Fre2, Pha2). For the brain control instructions that obtain the same frequency-phase information, the flashing rules of their corresponding EEG stimulations are the same.

[0116] The present invention reuses frequency and phase information of visual stimulation and can be applied to a variety of visual stimulation paradigms, both in low-frequency and high-frequency situations. It solves the problem of limited number of stimulation coding combinations in the classification of large EEG instructions, especially in high-frequency stimulation with high comfort. When regional information can be accurately divided, the method can greatly reduce the number of EEG stimulation classifications and ensure the optimal EEG stimulation classification results.

[0117] At time S15.t, the user gazes at a certain brain control instruction, and the flickering pattern of the electroencephalogram stimulation corresponding to the brain control instruction induces an electroencephalogram signal feature. Based on the electroencephalogram signal feature and the first corresponding relationship, a corresponding electroencephalogram stimulation classification is obtained;

[0118] As an example, in the case of dynamic partitioning where the position of the brain control command is not fixed, assuming that the EEG signal feature induced by the flickering law of the EEG stimulation corresponding to the brain control command that the user is watching is feature 1, according to Table 1, the EEG stimulation classification corresponding to the EEG signal feature is U1, because the frequency-phase information of the U1 type EEG stimulation is (Fre1, Pha1). Figure 6 The brain control instructions with frequency-phase information (Fre1, Pha1) are: A, E, I.

[0119] S16. Obtaining the pixel coordinates of the point of sight on the interactive interface when the user is looking at a certain brain control command at time t, and determining the sub-area where the point of sight is located;

[0120] S17. Based on the EEG stimulation classification obtained in S15 and the sub-areas determined in S16, the brain control instructions that the user needs to execute are determined and output to achieve dynamic brain-computer interaction.

[0121] As an example, assuming that when the user looks at the brain control command, the pixel coordinates of the line of sight on the interactive interface are in the second sub-area. Since among the brain control commands A, E and I, E is in the second sub-area, it can be determined that the brain control command the user wants to output is E.

[0122] The same method can also be used in the case of dynamic partitioning where the position of the brain control command is fixed to determine the brain control command that the user wants to output, which will not be elaborated here.

[0123] When the sub-area position needs to be fixed as preset, see Figure 8, there are 5 fixed sub - regions in the interaction interface (the 6 brain - control instructions within the solid line frame of sub - region 4 and the 4 brain - control instructions within the dashed line frame appear alternately according to different time sequences). Assume that in this case, according to the user's offline EEG experiment, the maximum number of EEG stimulation classifications for the user to achieve the optimal EEG stimulation classification effect is 6, and the number of brain - control instructions in each sub - region does not exceed 6. Then, there is no need to perform the partitioning operation, and frequency - phase information can be assigned to each brain - control instruction to continue with the subsequent steps. If the number of brain - control instructions in a certain sub - region exceeds 6, then partition based on the actual requirement by fixing or not fixing the positions of the brain - control instructions, and perform the corresponding dynamic partitioning steps.

[0124] In a second aspect, the present invention provides a terminal, including a processor and a communication interface coupled to the processor. The processor is used to run computer programs or instructions to implement the multi - interface fusion dynamic brain - machine interaction method based on frequency - phase information multiplexing provided in the first aspect.

[0125] To describe the technical solution of the present invention more clearly, the following will be further elaborated with specific examples.

[0126] Suppose there are 2 interfaces, and the models of the display devices presenting these 2 interfaces are the same, both being Dell Alienware 27 - inch monitors (resolution 2560*1440 pixels, refresh rate 240 Hz). Connect the 2 display devices to the same graphics card of the workstation host. The workstation host model is Intel Xeon 6226R Golden CPU@3.9 GHz, and the graphics card model is NVIDA3090. Set the unified device clock frequency through the graphics card surround to keep the interfaces refreshed synchronously and align the device timestamps. Merge the interaction interface canvases across multiple monitors and set it to the transparent penetration mode. According to the single - column arrangement mode of the display devices, the resolution of the interaction interface is calculated to be 2560*2880, as Figure 9 shown.

[0127] Based on the user's offline EEG experiment, the maximum number of EEG stimulation classifications for the user to achieve the optimal EEG stimulation classification effect is 4. Record the frequency - phase information of the 4 - classification EEG stimulation as follows: First classification: (31HZ, 0), Second classification: (32HZ, 0.35π), Third classification: (33HZ, 0.7π), Fourth classification: (34HZ, 1.05π). Record the first correspondence relationship between the 4 - classification EEG stimulation and the EEG signal characteristics as follows: First classification → Feature 1, Second classification → Feature 2, Third classification → Feature 3, Fourth classification → Feature 4.

[0128] Suppose at time t, 19 (to verify the method proposed in the present invention to the greatest extent, simulating the situation where a large number of brain - control instructions need to be presented in a complex scenario) brain - control instructions A - S need to be presented.

[0129] In the case of dynamic partitioning where the positions of brain control commands are not fixed, obtain the number of partitions The number of pixel points covered by each sub-region is calculated as follows: pixel points. Assuming pixel partitioning by column, the row pixel ranges of each sub-region can be obtained as (0, 2560]. The column pixel range of the first sub-region is (0, 576], the column pixel range of the second sub-region is (1×576, 2×576], the column pixel range of the third sub-region is (2×576, 3×576], and so on.

[0130] Allocate 19 brain control commands to 5 sub-regions in sequence. The number of brain control commands in the first 4 sub-regions is 4 each, and the number of brain control commands in the 5th sub-region is 3. Assuming the brain control commands are square, the side length l and the spacing d of the brain control commands are both 200 pixels. The central position coordinates of each brain control command in each sub-region can be calculated, and the brain control commands are divided to the corresponding coordinate positions of the sub-regions. Allocate frequency-phase information to each brain control command in each sub-region. For the partitioning and allocation results, see Figure 10 .

[0131] In the case of dynamic partitioning where the positions of brain control commands are fixed, read the central position coordinates of these 19 brain control commands Among them, Arrange these 19 brain control commands in ascending order according to column pixels If the column pixels are the same, then arrange them in ascending order according to row pixels The obtained order of brain control commands is: ADLKENRSHJIBCFGHMOP. Select the first 4 brain control commands ADLK as the first sub-region. Assuming that among the central position coordinates of these 4 brain control commands, the minimum row pixel coordinate is 450, the maximum is 2000, the minimum column pixel coordinate is 280, the maximum is 480, and the side length l of the brain control command is 200, then the row pixel range of this sub-region is [250, 2200), and the column pixel range is [80, 680). To avoid overlapping of sub-regions, it is necessary to judge whether there is an overlap between the pixel range of this sub-region and the pixel range of the 5th brain control command E during the partitioning process. If there is no overlap, the brain control commands within this sub-region are ADLK; if there is an overlap, then take ADL as the first sub-region and repeat the above partitioning process. Using this iterative partitioning strategy can ensure good classification results. Allocate frequency-phase information to each brain control command in each sub-region. For the partitioning and allocation results, see Figure 11 .

[0132] For the interactive interface with dynamic partitioning completed, the frequency-phase information obtained by brain control commands with the same order in each sub-region is the same, and the frequency-phase information obtained by each brain control command in a single sub-region is independent of each other. Figure 10 andFigure 11 Among them, the brain control instructions with the same frequency-phase information are marked with the same gray scale.

[0133] At time t, the user gazes at a brain control instruction, and the flashing pattern of the corresponding electroencephalogram (EEG) stimulation of this brain control instruction induces the characteristics of the EEG signal. Assuming that the induced characteristics of the EEG signal are Feature 2, it can be known that the EEG stimulation corresponding to this EEG signal characteristic is classified as the second classification. Taking Figure 11 the shown partition result as an example, it can be known that the brain control instructions corresponding to the EEG stimulations belonging to the second classification are D, N, I, C, H, O. Assuming that the pixel coordinates of the gaze landing point on the interaction interface when the user gazes at this brain control instruction are in the 4th sub-region, it can be determined that the brain control instruction the user wants to execute is C; assuming that the pixel coordinates of the gaze landing point on the interaction interface when the user gazes at this brain control instruction are in the 2nd sub-region, it can be determined that the brain control instruction the user wants to execute is N.

[0134] Although the present invention has been described in connection with various embodiments, however, in the process of implementing the claimed invention, those skilled in the art can understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure content, and the like. In the specification, the term "comprising" does not exclude other components or steps, and "a" or "one" does not exclude the case of multiple. A single processor or other unit can implement several functions listed in the specification. Certain measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0135] Although the present invention has been described in connection with specific features and their embodiments, it is obvious that various modifications and combinations can be made without departing from the spirit and scope of the present invention. Accordingly, this specification and the drawings are only exemplary descriptions of the present invention, and are considered to have covered any and all modifications, variations, combinations, or equivalents within the scope of the present invention. Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalent technologies, the present invention also intends to include these changes and modifications.

Claims

1. A multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing, characterized in that, Configure an offline EEG experiment for the user to obtain the maximum number of EEG classifications U for achieving the optimal EEG classification effect for the user, the frequency-phase information of the U-classified EEG stimulations, and the first correspondence between the U-classified EEG stimulations and the EEG signal features; The dynamic brain-computer interaction method comprises: S10. Merging multiple interfaces into one interactive interface, aligning the clock frequencies of the multiple interfaces, and unifying the EEG tag timestamps of the interactive interfaces; S11. Merging the resolutions of the multiple interfaces to obtain the resolution of the interactive interface; S12. When the brain control instructions change over time, configure N brain control instructions that need to be presented at a certain time t, assign brain electrical stimulation to the N brain control instructions, and obtain a second correspondence between the brain control instructions and the brain electrical stimulation; S13. Dynamically partitioning the interactive interface based on the resolution of the interactive interface and the maximum number of brain electrical stimulation categories U to obtain Z sub-areas and brain control instructions included in each sub-area; S14. Assign the frequency-phase information of the U-classified electroencephalographic stimulation to each brain control instruction in each sub-region. At this time, the electroencephalographic stimulation corresponding to each brain control instruction in the sub-region flashes according to the assigned frequency-phase information, and obtain the flashing law of the electroencephalographic stimulation; At time S15.t, the user gazes at a certain brain control instruction, and the flickering pattern of the electroencephalogram stimulation corresponding to the brain control instruction induces an electroencephalogram signal feature, and based on the electroencephalogram signal feature and the first corresponding relationship, a corresponding electroencephalogram stimulation classification is obtained; S16. Obtaining the pixel coordinates of the point of sight on the interactive interface when the user is gazing at the brain control command at time t, and determining the sub-area where the point of sight is located; S17. Based on the EEG stimulation classification obtained in S15 and the sub-areas determined in S16, the brain control instructions that the user needs to execute are determined and output to achieve dynamic brain-computer interaction.

2. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 1, characterized in that Specifically, S14 includes allocating the frequency-phase information of the U-type electroencephalographic stimulation to the brain control instructions in each sub-area in ascending order of the coordinates of the brain control instructions in each sub-area, and the frequency-phase information corresponding to the brain control instructions in the same sub-area is not repeated.

3. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 1, characterized in that, The resolution of the interactive interface is obtained by the following method: When the multiple interfaces are arranged in a single column, the resolution of the interaction interface is [X, Y], where X = R i , where R i is the minimum number of row pixels in the multiple interfaces, C k is the number of column pixels of the k-th interface, M is the total number of interfaces, X is the total number of row pixels of the interaction interface, and Y is the total number of column pixels of the interaction interface; When multiple interfaces are arranged in a single row, the resolution of the interactive interface is [X, Y], Y = C i ; where, R k is the number of row pixels of the k-th interface, C i is the minimum number of column pixels in the multiple interfaces, M is the total number of interfaces, X is the total number of row pixels of the interactive interface, and Y is the total number of column pixels of the interactive interface.

4. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 1, wherein S13 includes: dynamic partitioning of brain control command positions that are not fixed, specifically: S130. dividing the number of brain control instructions N by the maximum number of brain electrical stimulation categories U, and rounding up the quotient to obtain Z sub-areas; S131. Allocate the N brain control instructions to the Z sub-regions in sequence, the number of brain control instructions in the first Z-1 sub-regions is U, and the number of brain control instructions in the Zth sub-region is NU×(Z-1).

5. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 4, wherein The S131 includes the following sub-steps: S1310. Obtain the number of pixel points covered by each sub-region based on the resolution of the interaction interface and the number of sub-regions Z: Z i represents the number of pixel points covered by the i-th sub-region, where i = 1, 2,..., Z; S1311. If the interactive interface is partitioned by row pixels, the row pixel range of each sub-region is The column pixel range is (0, Y]; If the interactive interface is partitioned by column pixels, the row pixel range of each sub-region is (0, X], and the column pixel range is where X is the total number of row pixels of the interactive interface, Y is the total number of column pixels of the interactive interface, Z is the number of sub-regions, i represents the i-th sub-region, and i = 1, 2,..., Z; S1312. Determine the center position coordinates of each brain control command in each sub-area, specifically: If the interactive interface is partitioned by row pixels, the central position coordinates of the v-th brain control instruction i in the i-th sub-region are: v as follows: If the interactive interface is partitioned by column pixels, the central position coordinates of the v-th brain control instruction i in the i-th sub-region are: v ​ where i represents the i-th sub-region, i ≤ Z, and i v represents the v-th brain control instruction in the i-th sub-region, S i represents the number of brain control instructions in the i-th sub-region. The brain control instructions are square, l represents the side length of the brain control instruction, and d represents the spacing between brain control instructions.

6. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 1, wherein S13 includes: dynamic partitioning with fixed brain control command positions. At this time, the center position coordinates of each brain control command are known, specifically: S133. Read the center position coordinates of each brain control instruction, and sort the N brain control instructions according to their center position coordinates; S134. For the i-th sub-region, determine the number of brain control instructions to be partitioned and the size of U; when the number of brain control instructions to be partitioned is greater than U, execute S135; when the number of brain control instructions to be partitioned is less than or equal to U, execute S136; S135. Divide the first U brain control instructions among the brain control instructions to be partitioned into the same sub-region, and determine whether the pixel range of this sub-region overlaps with the pixel range of the (U + 1)-th brain control instruction among the brain control instructions to be partitioned. If not, the number of brain control instructions in this sub-region is U, let i = i + 1 and continue to execute S134; otherwise, let U = U - 1 and re-execute S135; S136. Divide the brain control instructions to be partitioned into the same sub-region, complete the dynamic partitioning, and obtain Z sub-regions and the brain control instructions included in each sub-region.

7. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 6, wherein The pixel range of the sub-region described in S135 is obtained based on the center coordinates of the brain control instructions partitioned into this sub-region, specifically: S1350. Obtain the minimum value x of the row pixel coordinates and the maximum value x of the central position coordinates of the brain control instructions divided into this sub-region min and the maximum value x max , and the minimum value y of the column pixel coordinates min and the maximum value y max ; S1351. The row pixel range of the sub-region is [x min -l, x max +l), and the column pixel range is [y min -l, y max +l); where the brain control instruction is square, and l represents the side length of the brain control instruction.

8. The multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing according to claim 6, characterized in that S133 specifically is: Read the center position coordinates of each brain control instruction, arrange the N brain control instructions in ascending order of the column pixel coordinates in their center position coordinates, and when the column pixel coordinates are the same, arrange them in ascending order of the horizontal pixel coordinates.

9. A terminal, characterized in that, It includes a processor and a communication interface coupled to the processor. The processor is used to run computer programs or instructions to implement the multi-interface fusion dynamic brain-computer interaction method based on frequency-phase information multiplexing described in any one of claims 1 to 8.

Images