Signal processing method based on brain control system, brain control system, device and equipment

Abstract

The embodiment of the present disclosure provides a signal processing method based on a brain control system, a brain control system, a device and equipment, and relates to the technical field of brain-computer interface. The method comprises the following steps: acquiring a user electroencephalogram under the condition that a target interface is displayed at a stimulation end; sending a to-be-executed instruction determined based on the user electroencephalogram to a controlled device, so that the controlled device acquires a second image in real time in the process of executing an operation corresponding to the to-be-executed instruction, and sends the second image to the stimulation end; receiving the second image sent by the controlled device; determining an execution result of the operation based on the second image, and updating the second image to the first image and returning to the step of acquiring the user electroencephalogram under the condition that the target interface is displayed at the stimulation end if the execution result does not meet a preset condition, until the execution result meets the preset condition. In this way, the visual needs of the user in the brain control related device scene can be effectively met.

Description

Technical Field

[0001] This disclosure relates to the field of brain-computer interface technology, and in particular to a signal processing method, brain control system, device and equipment based on a brain control system. Background Technology

[0002] Brain-computer interface (BCI) technology enables direct communication between the brain and external devices without relying on muscles and peripheral nerves. Current mainstream paradigms include the steady-state visual evoked potential (SSVEP) paradigm, which controls external devices by recognizing the brain's visual cortex's response to visual stimuli of specific frequencies (such as flashing lights or moving images). However, because SSVEP signals have the strongest response in the low-to-mid frequency range, and the fovea of ​​the retina has visual advantage, most SSVEP-based BCI systems use low-to-mid frequency flashing stimuli that easily lead to visual fatigue. Furthermore, existing SSVEP-based BCI systems typically present stimuli in the central visual field, which exacerbates visual discomfort and encroaches on visual space used to display crucial feedback information. Thus, existing SSVEP-based BCI systems struggle to meet users' visual needs, negatively impacting the user experience. Summary of the Invention

[0003] This disclosure provides a signal processing method, brain control system, device, and equipment based on a brain control system, in order to at least solve the above-mentioned technical problems existing in the prior art.

[0004] In a first aspect, embodiments of this disclosure provide a signal processing method based on a brain-controlled system, applied to the stimulation end of the brain-controlled system, the method comprising:

[0005] With the target interface displayed on the stimulation end, the user's EEG signal is acquired; the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target; the user's EEG signal is generated based on the flashing induced by the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets.

[0006] Send the instruction to be executed based on the user's EEG signal to the controlled device, so that the controlled device can acquire a second image in real time and send the second image to the stimulation end during the operation corresponding to the instruction to be executed;

[0007] Receive the second image sent by the controlled device;

[0008] The execution result of the operation is determined based on the second image. If the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions.

[0009] Secondly, embodiments of this disclosure provide a signal processing method based on a brain-controlled system, applied to a controlled device within the brain-controlled system, the method comprising:

[0010] The system receives an execution command determined based on the user's EEG signal from the stimulation terminal. The central area of ​​the target interface displayed on the stimulation terminal is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target. The user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets.

[0011] During the execution of the operation corresponding to the instruction to be executed, the second image is acquired in real time;

[0012] A second image is sent to the stimulation end so that the stimulation end can determine the execution result of the operation performed by the controlled device based on the second image. If the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions.

[0013] Thirdly, this disclosure provides a brain control system, which includes: a data acquisition end, a stimulation end, and a controlled device, wherein the data acquisition end is communicatively connected to the stimulation end, and the stimulation end is communicatively connected to the controlled device;

[0014] The acquisition end is used to acquire the user's EEG signals when the target interface is displayed on the stimulation end, and to feed back the user's EEG signals to the stimulation end; wherein, the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets;

[0015] The stimulation end is used to send the execution instructions to the controlled device based on the user's EEG signals;

[0016] The controlled device is used to acquire a second image in real time and send the second image to the stimulation end during the execution of the operation corresponding to the instruction to be executed;

[0017] The stimulation end is also used to receive a second image sent by the controlled device; determine the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image, and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops.

[0018] Fourthly, embodiments of this disclosure provide a signal processing device based on a brain-controlled system, applied to the stimulation end of the brain-controlled system, the device comprising:

[0019] The acquisition module is used to acquire the user's EEG signal when the target interface is displayed on the stimulation end; the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets.

[0020] The sending module is used to send the instruction to be executed based on the user's EEG signal to the controlled device, so that the controlled device can acquire the second image in real time and send the second image to the stimulation end during the operation corresponding to the instruction to be executed.

[0021] The receiving module is used to receive the second image sent by the controlled device;

[0022] The determination module is used to determine the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops.

[0023] Fifthly, embodiments of this disclosure provide a signal processing device based on a brain-controlled system, applied to a controlled device, the device comprising:

[0024] The receiving module is used to receive the execution command determined based on the user's EEG signal sent by the stimulation end. The central area of ​​the target interface displayed by the stimulation end is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target. The user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets.

[0025] The execution module is used to acquire the second image in real time during the execution of the operation corresponding to the instruction to be executed;

[0026] The sending module is used to send a second image to the stimulation end, so that the stimulation end can determine the execution result of the operation performed by the controlled device based on the second image. If the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions.

[0027] In a sixth aspect, embodiments of this disclosure provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the signal processing method based on the brain control system of the first aspect or the second aspect.

[0028] In a seventh aspect, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a signal processing method based on a brain-controlled system according to the first or second aspect.

[0029] Based on the signal processing method of the brain-controlled system provided in this disclosure, the user's electroencephalogram (EEG) signal, generated by the flashing area corresponding to the command target, can be acquired through the stimulation end when the user gazes at the edge area of ​​the target interface and commands a target. The controlled device can then transmit the image back to the central area of ​​the target interface in real time for display during the execution of the command determined based on the user's EEG signal, forming a visual feedback closed loop. In this process, the central area of ​​the target interface displayed by the stimulation end can display the first image acquired in real time by the controlled device, while the edge area of ​​the target interface can be used to display multiple command targets and the flashing area associated with each command target. This effectively avoids visual fatigue caused by the user directly viewing low-frequency flashing stimuli and allows the first image acquired in real time by the controlled device to be displayed in the central area of ​​the interface, preventing the visual space used for displaying key feedback information from being squeezed out, thus satisfying the user's visual needs and improving the user experience.

[0030] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the structure of a brain control system provided in an embodiment of this disclosure;

[0032] Figure 2 This is a flowchart illustrating the interaction between a stimulation end and a controlled device provided in an embodiment of this disclosure;

[0033] Figure 3 This is a scenario example diagram of a signal processing method based on a brain control system provided in an embodiment of this disclosure;

[0034] Figure 4 This is an example diagram of a target interface provided in an embodiment of this disclosure;

[0035] Figure 5 This is one of the structural schematic diagrams of a signal processing device based on a brain control system provided in this disclosure embodiment;

[0036] Figure 6 This is a second schematic diagram of the structure of a signal processing device based on a brain control system provided in this embodiment of the present disclosure;

[0037] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0038] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0039] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0042] As described in the background section, brain-computer interface technology can directly establish an information transmission channel for interaction between the brain and external devices (such as drones) without relying on muscles and peripheral nerves, thus enabling direct communication between the brain and external devices.

[0043] Currently, the mainstream SSVEP paradigm utilizes the frequency domain consistency between visual stimulus frequencies and the response frequencies of the brain's visual cortex. It is a high-speed brain-computer interface (BCI) paradigm with advantages such as short system calibration time, minimal training requirements, and a large number of coded signals. However, because SSVEP signals have the strongest response in the low-to-mid frequency range, and the fovea of ​​the retina has a visual advantage, most SSVEP-based BCI systems use low-to-mid frequency flickering stimuli that easily lead to visual fatigue. Furthermore, existing SSVEP-based BCI systems typically present stimuli in the central visual field, which exacerbates visual discomfort and encroaches on visual space used to display crucial feedback information. Consequently, existing SSVEP-based BCI systems struggle to meet users' visual needs, thus impacting the user experience.

[0044] Based on this, the present disclosure provides a signal processing method based on a brain control system to at least solve the technical problem that existing SSVEP-based brain-computer interface systems are unable to meet users' visual needs, thereby affecting the user experience.

[0045] It should be noted that, since the signal processing method based on a brain control system provided in this disclosure involves a brain control system, in order to facilitate a better understanding of the signal processing method based on a brain control system provided in this disclosure, the appendix will be consulted before introducing the signal processing method based on a brain control system provided in this disclosure. Figure 1 The mind control system involved in the embodiments of this disclosure will be described in detail.

[0046] Figure 1 This is a schematic diagram of the structure of a brain control system provided in an embodiment of this disclosure.

[0047] like Figure 1 As shown, the brain control system 100 may include a data acquisition terminal 11, a stimulation terminal 12, and a controlled device 13. It should be noted that the data acquisition terminal 11 is communicatively connected to the stimulation terminal 12, and the stimulation terminal 12 is communicatively connected to the controlled device 13. The transmission mode between the stimulation terminal 12 and the controlled device 13 can be either half-duplex or full-duplex, and the communication method between the stimulation terminal 12 and the controlled device 13 can be a wireless medium-to-long-range data transmission method; no specific limitation is made here.

[0048] The acquisition end 11 is used to acquire the user's EEG signal when the target interface is displayed on the stimulation end, and to feed back the user's EEG signal to the stimulation end; the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets;

[0049] Stimulation terminal 12 is used to send the instruction to be executed to the controlled device based on the user's EEG signal;

[0050] The controlled device 13 is used to acquire a second image in real time and send the second image to the stimulation end during the execution of the operation corresponding to the instruction to be executed;

[0051] The stimulation end 12 is also used to receive a second image sent by the controlled device; determine the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image, and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops.

[0052] Based on the above Figure 1 The brain control system shown below, with reference to the accompanying drawings, provides a detailed description of the signal processing method based on the brain control system provided in this disclosure through specific embodiments.

[0053] Figure 2 This is a flowchart illustrating the interaction between a stimulation end and a controlled device provided in an embodiment of this disclosure.

[0054] like Figure 2 As shown in the embodiments of this disclosure, the signal processing method based on a brain control system may specifically include the following steps:

[0055] S210, the stimulation end acquires the user's EEG signal while the target interface is displayed on the stimulation end.

[0056] In some embodiments, the central area of ​​the target interface can be used to display a first image acquired in real time by the controlled device, wherein the controlled device may be, for example, a drone, a robot, a vehicle, or other equipment, without being specifically limited here.

[0057] The edge area of ​​the aforementioned target interface can be used to display multiple command targets and multiple flashing areas. These flashing areas can include the flashing area associated with each command target. Each command target can indicate a command, which can control the controlled device to perform a corresponding action or operation. This action or operation requires the controlled device to move in three dimensions; that is, the command must include at least three degrees of freedom to ensure that the controlled device can perform the corresponding action or operation in real three-dimensional space. For example, if the controlled device is a drone, the command can be used to control it to perform flight actions such as takeoff, landing, and movement. These actions are achieved by controlling the drone's movement in three dimensions, such as forward and backward translation along the X-axis, left and right translation along the Y-axis, ascent or descent along the Z-axis, and rotation around the Z-axis, without specific limitations here.

[0058] It should be noted that the aforementioned command targets themselves do not flash. However, each flashing region can flash at a specific frequency to induce corresponding EEG signals in the user's visual cortex. Thus, the user's EEG signals can be generated based on the flashing of the flashing region associated with the first command target when the user is focused on it. This avoids visual fatigue caused by directly looking at the flashing command target, which will not be elaborated further here. Furthermore, to further reduce the flickering sensation during system use, the flashing frequency of each flashing region can be set to a frequency higher than a preset flashing frequency. This avoids low-to-medium frequency flashing stimulation that easily leads to visual fatigue. The preset flashing frequency can be determined based on actual conditions; for example, it can be set to... No further restrictions are imposed here.

[0059] It should also be noted that the relationship between the number of command targets and the number of flashing areas depends on the interface layout and interaction requirements, and is not a fixed one-to-one correspondence. Each command target can be associated with one or more flashing areas. For example, when command targets and flashing areas are arranged alternately along the edge of the interface, the number of command targets can be set to n (n≥1), and the number of flashing areas can be set to (n-1). Of course, the number of flashing areas can also be set to n or more (e.g., m, m>n), and no specific limitation is made here.

[0060] S220, the stimulating end sends the instruction to be executed to the controlled device based on the user's EEG signal.

[0061] S230: The controlled device acquires a second image in real time while executing the operation corresponding to the instruction to be executed.

[0062] The second image is different from the first image mentioned above, and no specific limitation is made here.

[0063] S240, the controlled device sends a second image to the stimulation end.

[0064] S250, the stimulation end determines the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions.

[0065] The execution results mentioned above may include the status information of the controlled device after the corresponding operation is completed, such as pose and execution success indicator, which will not be elaborated on here. The preset conditions can be pre-set based on task requirements. For example, the preset conditions can be set to whether the execution result meets the task execution objective. The task instruction objective can be determined based on the task instruction pre-acquired by the stimulation end, which is not specifically limited here.

[0066] Specifically, when the stimulation terminal displays the target interface, the user can focus on any one of the multiple instruction targets (i.e., the first instruction target) presented in the edge area of ​​the target interface, based on the first image presented in the central area of ​​the target interface. During the user's focus on the first instruction target, the flashing of the flashing area associated with the first instruction target can induce the user's visual cortex to generate corresponding electroencephalogram (EEG) signals (i.e., the user's EEG signals). In this way, the stimulation terminal can acquire the user's EEG signals and send the instruction to be executed to the controlled device based on the user's EEG signals. Then, after receiving the instruction to be executed, the controlled device can execute the operation corresponding to the instruction to be executed, and during the execution of the operation, it can acquire a second image in real time and feed the second image back to the stimulation terminal. In this way, the stimulation terminal can determine the execution result of the operation performed by the controlled device based on the second image and determine whether the execution result meets the preset conditions. If the execution result does not meet the preset conditions, the stimulation terminal can return to execution S210 until the execution result meets the preset conditions and then stops.

[0067] In one example, such as Figure 3 As shown, the controlled device can transmit the acquired real-time images back to the stimulation end in real time, and... Figure 3 The target interface of the stimulation terminal is clearly displayed. Based on this, the acquisition terminal can simultaneously acquire the user's EEG signals and then send the user's EEG signals to the stimulation terminal. The stimulation terminal then sends the execution command determined based on the user's EEG signals to the controlled device, realizing precise control of the controlled device. Subsequently, the above process is repeated, continuously performing operations such as screen feedback, signal acquisition, processing, and command transmission until the execution result meets the preset conditions.

[0068] It should also be noted that, in this embodiment, each trial of online control may include a visual shift time and a flashing time. The duration of the visual shift time and the flashing time can be determined according to the actual situation. For example, the visual shift time can be set to 3 seconds and the flashing time can be set to 2.6 seconds. During the visual shift time, the flashing area turns green, and the user needs to select the instruction to be sent based on the real-time image captured by the drone and visually shift to the instruction target corresponding to that instruction. During the flashing time, the flashing area begins to flash, and the user continues to look at the instruction target. At the end of a trial, the stimulation end analyzes the real-time acquired user EEG signal and provides the instruction to be executed corresponding to the user's EEG signal. The drone executes the operation corresponding to the instruction to be executed and transmits back the captured real-time image in real time.

[0069] In this embodiment, the user's electroencephalogram (EEG) signal, generated by the flashing area corresponding to the command target, can be acquired via the stimulation end when the user gazes at the edge area of ​​the target interface. The controlled device can then transmit the image back to the central area of ​​the target interface in real time during the execution of the command determined based on the user's EEG signal, forming a visual feedback closed loop. During this process, the central area of ​​the target interface displayed by the stimulation end can display the first image acquired in real time by the controlled device, while the edge area of ​​the target interface can be used to display multiple command targets and the flashing area associated with each command target. This effectively avoids visual fatigue caused by the user directly viewing low-frequency flashing stimuli and allows the first image acquired in real time by the controlled device to be displayed in the central area of ​​the interface, preventing the visual space used for displaying key feedback information from being squeezed out, thus satisfying the user's visual needs and improving the user experience.

[0070] In order to accurately and comprehensively describe the signal processing method based on the brain control system provided in the embodiments of this disclosure, in one embodiment, the signal processing method based on the brain control system provided in the embodiments of this disclosure may further include the following steps:

[0071] Obtain the interface layout information of the target interface;

[0072] Based on the interface layout information, the central area and edge area in the target interface are determined, and the display area and multiple flashing areas corresponding to multiple instruction targets in the edge area are determined respectively, so that the central area, the display area and multiple flashing areas corresponding to multiple instruction targets do not overlap.

[0073] The interface layout information of the target interface may include relevant information about the interface design or layout of the target interface. As can be seen from the content of the foregoing embodiments, the target interface can be divided into a central area and an edge area. The edge area may include multiple command targets and a flashing area associated with each command target. That is, the edge area can be divided into display areas corresponding to multiple command targets and flashing areas associated with each command target. Therefore, the interface layout information may include relevant area information of the central area, edge area, display area corresponding to the command target, and flashing area. No specific limitation is made here.

[0074] In this way, the stimulation end can obtain the interface layout information of the target interface, and determine the central area and edge area of ​​the target interface based on the interface layout information, and determine the display area and multiple flashing areas corresponding to multiple instruction targets in the edge area, so that the central area, the display area and multiple flashing areas corresponding to the multiple instruction targets do not overlap.

[0075] In one example, if the target interface is displayed on an LCD screen with a resolution of 1920×1080 pixels and a refresh rate of 120Hz, the stimulation device can obtain the interface layout information of the target interface and, based on this layout information, determine the display size of the display areas corresponding to the 12 command targets used to indicate commands such as ascend, move left, turn left, descend, take off, move forward, move backward, land, take a picture, turn right, move right, and exit, as well as the display size of the 11 flashing areas. The display size of the display areas corresponding to the command targets is 120×120 pixels, and the display size of the flashing areas is 120×90 pixels. Based on this, assuming the origin of the coordinate system is located at the top left corner of the target interface, the positive x-axis is horizontal to the right, and the positive y-axis is vertical to the bottom, the coordinates of the top left corner of the 12 command targets can be set as (0, 0), (0, 290), (0, 580), (0, 870), (360, 870), (720, 870), (1080, 870), (1440, 870), (1800, 870), (1800, 580), (1800, 290), (1800, 0). The coordinates of the top-left corners of the 11 flashing areas can be set to (0, 160), (0, 450), (0, 740), (180, 990), (540, 990), (900, 990), (1260, 990), (1620, 990), (1800, 740), (1800, 450), and (1800, 160), respectively. Additionally, the display size of the central area used to display the real-time image can be 960×720 pixels, and the coordinates of the top-left corner of this central area on the target interface are (480, 0). The approximate layout of the display area, flashing areas, and central area corresponding to the above instruction target can be as follows: Figure 4As shown, no further restrictions are imposed here.

[0076] It should also be noted that the interface layout information of the aforementioned target interface may also include the colors of the target interface, the display area corresponding to the instruction target, the blinking area, and the central area. For example, the background of the target interface can be set to a grayscale value of 102, and the display area of ​​the instruction target can be set to a grayscale value of 0. In addition, the interface layout information of the aforementioned target interface may also include text color and text size. For example, the grayscale value of the instruction text displayed on the instruction target can be set to 255, which will not be elaborated further here. Furthermore, since the instruction target needs to be labeled with its corresponding instruction, the text size of the instruction can be adjusted according to the actual dots per inch (DPI) of the display to ensure that the instruction is contained within the instruction target.

[0077] In this embodiment, the interface layout information of the target interface can be obtained in advance, and the interface layout of the target interface can be realized based on the interface layout information. This ensures that the display areas and flashing areas of multiple instruction targets in the central area and edge area of ​​the target interface do not overlap, thereby effectively avoiding mutual occlusion of the interface information of the target interface, improving visual clarity, and meeting the user's visual needs.

[0078] Furthermore, considering that if the display areas and flashing areas corresponding to multiple instruction targets within the edge region are too small, it may result in the user being unable to clearly focus on the instruction target corresponding to the instruction they want to select, or the display areas or flashing areas corresponding to each instruction target being too close together, making it impossible to accurately determine the instruction to be executed based on the user's EEG signals. Therefore, in some embodiments, the aforementioned interface layout information may include the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target, etc., which are not specifically limited here.

[0079] Therefore, the steps described above for obtaining the interface layout information of the target interface can specifically include the following steps:

[0080] Determine the target distance from the user's eye position point to the target interface;

[0081] Based on the target distance, calculate the relative position between each command target and the flashing area associated with the command target. The relative position includes the relative angle and the relative distance.

[0082] Based on the relative position between each instruction target and the flashing area associated with the instruction target, the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target are calculated.

[0083] The user's eye location can be any location within the user's eye area, such as the left eye location, right eye location, corner of the eye location, or center of the eye location; no specific limitation is made here. Additionally, the aforementioned relative positions can include relative angles and relative distances, which are also not specifically limited here.

[0084] Specifically, the stimulation end can determine the target distance between the user's eye position and the target interface in real time, and can calculate the relative position between each instruction target and the flashing area associated with the instruction target based on the target distance. Since the relative position may include a relative angle, the stimulation end can then calculate the display size of the instruction target and the size of the flashing area associated with the instruction target based on the relative position between each instruction target and the flashing area associated with the instruction target.

[0085] In one example, if the distance between the user's eye position point acquired by the stimulus end and the target distance on the target interface is... Then, the target distance can be determined as... Determine the relative angle between each command target and the flashing area. The relative angle It can be greater than 2°, depending on the relative angle. The distance D between the target and the command target displayed on the target interface is calculated to determine the relative distance between the command target and the flashing area. For example, relative angles The distance D between the target and the display area corresponding to the command target is determined by the relative distance between the display area and the flashing area. It can satisfy d=D The relationship presented here is not specifically limited, and the length of the display area corresponding to the instruction target can be determined based on the relative position between the two. ,Width With the length of the flashing area ,Width It should be noted that, in order to avoid the number of command targets and flashing areas in the same direction being too large and exceeding the display range, constraints can be set to limit the layout, thereby preventing the sum of the resolution, spacing, etc. of command targets and flashing areas in the same direction from exceeding the original resolution of the target interface. The preset conditions can be determined according to the specific layout of the target interface, and no specific limitation is made here.

[0086] It should also be noted that the target distance and the relative angle between the command target and the flashing area, the relative position and the display size of the display area corresponding to the command target, and the display size of the flashing area all satisfy certain mapping relationships. This disclosure embodiment does not specifically limit the mapping relationships involved here.

[0087] In this embodiment, by acquiring the target distance between the current user's key eye points and the target interface, the display size of the display area and the flashing area corresponding to the instruction target that meets the user's visual needs, as well as the relative distance between them, can be dynamically determined. This can effectively prevent situations where the user cannot clearly focus on the instruction target corresponding to the instruction they want to select, or where the display area or flashing area corresponding to each instruction target is too close, resulting in the inability to accurately determine the instruction to be executed based on the user's EEG signal. It can also effectively prevent the mutual occlusion of interface information on the target interface, improve visual clarity, and meet the user's visual needs.

[0088] In order to provide a comprehensive and detailed description of the signal processing method based on a brain control system provided in this disclosure, in one embodiment, before S220, the signal processing method based on a brain control system provided in this disclosure may further include the following steps:

[0089] Determine the instructions to be executed based on the user's brainwave signals;

[0090] Based on the first image and the instruction to be executed, calculate the executable probability of the instruction to be executed;

[0091] If the probability of executing the instruction to be executed is less than the preset execution probability, the instruction to be executed is determined to be an erroneous instruction, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the probability of executing the instruction to be executed is greater than or equal to the preset execution probability.

[0092] The executable probability of the aforementioned instruction can be used to assess the likelihood of its successful execution. The preset execution probability can be a probability threshold pre-set based on practical experience or circumstances; no specific limitations are imposed here.

[0093] It should be noted that the executable probability of the above-mentioned instruction to be executed can be determined by analyzing the state information of the controlled device reflected in the first image and the instruction to be executed using a preset model or preset algorithm. Alternatively, the executable probability of the instruction to be executed can be determined based on the pre-set correspondence between the state information of the controlled device reflected in the first image, the instruction to be executed, and the executable probability. No specific limitation is made here.

[0094] Specifically, the stimulation end can acquire the user's EEG signal and determine the instruction to be executed based on the user's EEG signal. It can also calculate the executable probability of the instruction based on the first image currently displayed on the target interface and the instruction to be executed, and then determine whether the executable frequency of the instruction to be executed is less than the preset execution frequency. If the executable frequency of the instruction to be executed is less than the preset execution frequency, it means that the success rate of the instruction to be executed is low. Therefore, the instruction to be executed can be determined as an erroneous instruction, and the execution process returns to S210 until the executable probability of the instruction to be executed is greater than or equal to the preset execution probability, at which point the instruction to be executed is output.

[0095] In one example, if the drone is in a landing state, a landing prompt message "Landing, please select takeoff or exit" will be displayed at the bottom of the interface. In this trial, the user can only choose "takeoff" to perform the takeoff operation or "exit" to exit the interface. If the user makes an incorrect selection or the stimulus detects another command, the drone's status information (the drone is in a landing state) reflected in the first image displayed in the central area indicates that the probability of executing that command is 0. The drone remains in a landing state, will not execute the command, and will proceed to the next trial. In another example, after the takeoff command is executed, the drone will rise to approximately 50 centimeters above the ground and enter a hovering state. At this point, the landing prompt message disappears, and the user can select commands other than "takeoff" and "exit" and determine the probability of executing that command. While hovering, if the user incorrectly selects or the stimulus incorrectly identifies it as a "take-off" or "exit" command, the probability of executing the "take-off" or "exit" command is determined to be 0 based on the drone's status information (the drone is hovering) reflected in the first image displayed in the central area. The drone will remain hovering and proceed to the next trial. When the user selects the "land" command, the probability of executing the "land" command is determined to be 100% based on the drone's status information (the drone is hovering) reflected in the first image displayed in the central area. The drone will then perform a landing operation to end the hovering state and enter the landing state.

[0096] In this embodiment, based on the acquired user's EEG signal, the corresponding instruction to be executed can be initially determined. The probability of successful execution of the instruction is assessed by calculating its executable probability. If the executable probability is low, the instruction is identified as an erroneous instruction, and the process returns to re-determining the instruction until its executable probability exceeds a preset probability. This effectively avoids situations where the instruction cannot be executed or is an erroneous instruction.

[0097] Furthermore, since the user's EEG signal acquired by the stimulation end is the raw EEG signal acquired by the acquisition end, this raw EEG signal may include not only the SSVEP signal but also other EEG components, electromyography (EMG), electrooculography (EOG), and other noise. Therefore, in order to accurately determine the instruction to be executed corresponding to the user's EEG signal, in one embodiment, before S220, the signal processing method based on the brain control system provided in this disclosure may include the following steps:

[0098] For each instruction target among multiple instruction targets, the template signal corresponding to the instruction target is determined based on the flashing frequency of the flashing area associated with the instruction target, thus obtaining multiple template signals;

[0099] Calculate the similarity between the user's EEG signal and each template signal to obtain multiple similarity scores;

[0100] Based on multiple similarities, the template signal with the highest similarity to the user's EEG signal among multiple template signals is identified as the target template signal, and the instruction indicated by the first instruction target corresponding to the target template signal is identified as the instruction to be executed.

[0101] It should be noted here that different flashing areas can flash at different frequencies in order to induce different response signals in the user's visual cortex. For example, Figure 4 The 11 flashing areas can be set to flashing frequencies of 31Hz, 31.75Hz, 32.5Hz, 33.25Hz, 31.25Hz, 32Hz, 32.75Hz, 33.75Hz, 31.5Hz, 32.25Hz, and 33Hz respectively, without any specific limitation.

[0102] Thus, since the edge region of the target interface includes multiple instruction targets, for each instruction target, the template signal corresponding to that instruction target can be determined based on the flashing frequency of the flashing region associated with that instruction target, thereby obtaining multiple different template signals. Next, the stimulation end can calculate the similarity between the user's EEG signal and each template signal, thus obtaining multiple similarity scores. Then, based on these multiple similarity scores, the stimulation end can determine the template signal with the highest similarity to the user's EEG signal as the target template signal, and determine the instruction indicated by the first instruction target corresponding to this target template signal as the instruction to be executed.

[0103] In one example, task-discriminant component analysis (TDCA) demonstrates superior performance as a frequency identification method. In embodiments of this disclosure, using... Figure 4As shown in the example, each instruction target corresponds to one or two flashing areas, and the corresponding reference signal also changes accordingly, as detailed below:

[0104] (1)

[0105] (2)

[0106] (3)

[0107] (4)

[0108] in, The number of sampling points. Sampling frequency, For the blinking time, It represents the harmonic number. This is the template signal corresponding to the first instruction target. This is the template signal corresponding to the target of the 12th instruction. For the first Template signal corresponding to each instruction target . The flashing frequency corresponding to the k-th instruction target.

[0109] To accommodate the nonlinear processing characteristics of the human brain, gamma correction can be used instead of the inverse of this nonlinear transformation. Data augmentation is performed on the user's EEG signals during real-time control of the controlled device. The average training data and augmented test data for each class are passed through a spatiotemporal filter and then gamma-corrected, subsequently used... The norm measures the difference and is used to calculate the final classification result.

[0110] In this embodiment, for each instruction target, since different flashing regions flash at different flashing frequencies, a template signal corresponding to that instruction target can be determined based on the flashing frequency of the flashing region associated with each instruction target. This results in multiple template signals corresponding to different instruction targets. Furthermore, by calculating the similarity between the user's EEG signal and each template signal, multiple similarity scores can be obtained. Based on these similarity scores, the template signal with the highest similarity to the user's EEG signal can be identified as the target template signal. The instruction indicated by the first instruction target corresponding to this target template signal is then identified as the instruction to be executed. In this way, the instruction to be executed corresponding to the user's EEG signal can be accurately determined.

[0111] Based on the same inventive concept, this disclosure provides a signal processing device based on a brain-controlled system. This signal processing device can be applied to the stimulation end of a brain-controlled system, and can be specifically described in conjunction with the appendix. Figure 5 This disclosure provides a detailed description of a signal processing device based on a brain control system.

[0112] Figure 5 This is a schematic diagram of the structure of a signal processing device based on a brain control system provided in an embodiment of this disclosure.

[0113] like Figure 5 As shown, the signal processing device 500 based on the brain control system may include:

[0114] The acquisition module 510 is used to acquire the user's EEG signal when the target interface is displayed on the stimulation end; the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets.

[0115] The sending module 520 is used to send the instruction to be executed based on the user's EEG signal to the controlled device, so that the controlled device can acquire the second image in real time and send the second image to the stimulation end during the operation corresponding to the instruction to be executed.

[0116] Receiver module 530 is used to receive the second image sent by the controlled device;

[0117] The determination module 540 is used to determine the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops.

[0118] In one embodiment, the signal processing device based on a brain-controlled system provided in this disclosure may include:

[0119] The acquisition module is also used to acquire the interface layout information of the target interface;

[0120] The determination module is also used to determine the central area and edge area in the target interface based on the interface layout information, and to determine the display area and multiple flashing areas corresponding to multiple instruction targets in the edge area, so that the central area, the display areas corresponding to multiple instruction targets and the multiple flashing areas do not overlap.

[0121] In one embodiment, the interface layout information includes the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target; based on this, the signal processing device based on the brain control system provided in this disclosure embodiment may include:

[0122] The determination module is also used to determine the target distance between the user's eye position and the target interface;

[0123] The calculation module is used to calculate the relative position between each command target and the flashing area associated with the command target based on the target distance. The relative position includes the relative angle and the relative distance.

[0124] The calculation module is also used to calculate the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target based on the relative position between each instruction target and the flashing area associated with the instruction target.

[0125] In one embodiment, the signal processing device based on a brain-controlled system provided in this disclosure may include:

[0126] The determination module is also used to determine the instructions to be executed based on the user's EEG signals;

[0127] The calculation module is also used to calculate the executable probability of the instruction to be executed based on the first image and the instruction to be executed;

[0128] The determination module is also used to determine that the instruction to be executed is an erroneous instruction if the probability of its execution is less than the preset execution probability, and to return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the probability of the instruction to be executed is greater than or equal to the preset execution probability.

[0129] In one embodiment, different flashing regions flash at different flashing frequencies; based on this, the signal processing device based on a brain-controlled system provided in this disclosure includes:

[0130] The determination module is also used to determine the template signal corresponding to each instruction target among multiple instruction targets based on the flashing frequency of the flashing area associated with the instruction target, thereby obtaining multiple template signals;

[0131] The calculation module is used to calculate the similarity between the user's EEG signal and each template signal, and obtain multiple similarity results;

[0132] The determination module is also used to determine, based on multiple similarities, the template signal with the highest similarity to the user's EEG signal among multiple template signals as the target template signal, and to determine the instruction indicated by the first instruction target corresponding to the target template signal as the instruction to be executed.

[0133] It is understood that, when implementing the corresponding brain-controlled system-based signal processing method, the signal processing device provided in the above embodiments can allocate the above processing to different program modules as needed to complete all or part of the processing described above. Furthermore, the device and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0134] Based on the same inventive concept, this disclosure provides a signal processing device based on a brain-controlled system. This signal processing device can be applied to controlled devices, and can be specifically described in conjunction with the appendix. Figure 6 This disclosure provides a detailed description of a signal processing device based on a brain control system.

[0135] Figure 6 This is a schematic diagram of the structure of a signal processing device based on a brain control system provided in an embodiment of this disclosure.

[0136] like Figure 6 As shown, the signal processing device 600 based on the brain control system may include:

[0137] The receiving module 610 is used to receive the execution command determined based on the user's EEG signal sent by the stimulation end. The central area of ​​the target interface displayed by the stimulation end is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and the flashing area associated with each command target. The user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among multiple command targets.

[0138] The execution module 620 is used to acquire the second image in real time during the execution of the operation corresponding to the instruction to be executed;

[0139] The sending module 630 is used to send a second image to the stimulation end, so that the stimulation end can determine the execution result of the operation performed by the controlled device based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image, and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops.

[0140] It is understood that, when implementing the corresponding brain-controlled system-based signal processing method, the signal processing device provided in the above embodiments can allocate the above processing to different program modules as needed to complete all or part of the processing described above. Furthermore, the device and the corresponding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0141] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a signal processing method based on a brain-controlled system.

[0142] This application provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by a processor, the processor will execute the signal processing method based on a brain control system provided in this application.

[0143] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0144] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0145] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0146] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0147] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure; as shown below. Figure 7 As shown, the electronic device 70 includes: a processor 701, and a memory 702 communicatively connected to the processor 701; the memory 702 stores instructions executable by the processor 701. The instructions are executed by the processor 701 to enable the processor 701 to perform:

[0148] When the target interface is displayed at the stimulation end, the user's EEG signal is acquired; the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and a flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among the multiple command targets.

[0149] Send the instruction to be executed based on the user's EEG signal to the controlled device, so that the controlled device can acquire a second image in real time and send the second image to the stimulation end during the execution of the operation corresponding to the instruction to be executed;

[0150] Receive the second image sent by the controlled device;

[0151] The execution result of the operation is determined based on the second image. If the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions.

[0152] Alternatively, the instruction is executed by processor 701 to enable processor 701 to perform:

[0153] The device receives an execution command determined based on the user's EEG signal from a stimulation terminal. The central area of ​​the target interface displayed on the stimulation terminal is used to display a first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and a flashing area associated with each command target. The user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among the multiple command targets.

[0154] During the execution of the operation corresponding to the instruction to be executed, a second image is acquired in real time;

[0155] The second image is sent to the stimulation terminal so that the stimulation terminal can determine the execution result of the controlled device performing the operation based on the second image. If the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation terminal, until the execution result meets the preset conditions.

[0156] The electronic devices provided in the above embodiments and the corresponding signal processing methods based on brain control systems belong to the same concept. For details of their implementation, please refer to the method embodiments, which will not be repeated here.

[0157] In practical applications, electronic device 70 may further include at least one network interface 703. The various components in electronic device 70 are coupled together via a bus system 704. It is understood that the bus system 704 is used to implement communication between these components. In addition to a data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 All buses are labeled as bus system 704. There can be at least one processor 701 and at least one memory 702. Network interface 703 is used for wired or wireless communication between electronic device 70 and other devices.

[0158] The memory 702 in this embodiment is used to store various types of data to support the operation of the electronic device 70.

[0159] The methods disclosed in the above embodiments of this disclosure can be applied to processor 701, or implemented by processor 701. Processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 701 or by instructions in the form of software. The processor 701 may be a general-purpose processor, a digital signal processor (DSP, DiGital Signal Processor), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 701 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this disclosure can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 702. Processor 701 reads the information in memory 702 and, in conjunction with its hardware, completes the steps of the aforementioned signal processing method based on a brain control system.

[0160] In some embodiments, the electronic device 70 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.

[0161] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0162] In the above description, the term "some embodiments" refers to a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0163] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used in this disclosure is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0164] It should be understood that in the various embodiments of this disclosure, the sequence number of each implementation process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0165] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0166] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. A signal processing method based on a brain-controlled system, characterized in that, The method, applied to a stimulation terminal in a brain-controlled system, includes: When the target interface is displayed at the stimulation end, the user's EEG signal is acquired; the central area of ​​the target interface is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and a flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among the multiple command targets. Send the instruction to be executed based on the user's EEG signal to the controlled device, so that the controlled device can acquire a second image in real time and send the second image to the stimulation end during the execution of the operation corresponding to the instruction to be executed; Receive the second image sent by the controlled device; The execution result of the operation is determined based on the second image, and if the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions. The method further includes: Obtain the interface layout information of the target interface; Based on the interface layout information, the central area and edge area in the target interface are determined, and the display area and multiple flashing areas corresponding to multiple instruction targets in the edge area are determined respectively, so that the central area, the display area corresponding to the multiple instruction targets and the multiple flashing areas do not overlap with each other; The interface layout information includes the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target; The step of obtaining the interface layout information of the target interface includes: Determine the target distance between the user's eye position point and the target interface; Based on the target distance, the relative position between each command target and the flashing area associated with the command target is calculated, and the relative position includes the relative angle and the relative distance; Based on the relative position between each instruction target and the flashing area associated with the instruction target, the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target are calculated.

2. The method according to claim 1, characterized in that, Before sending the instruction to be executed, determined based on the user's EEG signal, to the controlled device, the method further includes: The instruction to be executed is determined based on the user's EEG signal; Based on the first image and the instruction to be executed, calculate the executable probability of the instruction to be executed; If the executable probability of the instruction to be executed is less than the preset execution probability, the instruction to be executed is determined to be an erroneous instruction, and the process returns to the step of acquiring the user's EEG signal when the target interface is displayed on the stimulation end, until the executable probability of the instruction to be executed is greater than or equal to the preset execution probability.

3. The method according to claim 1 or 2, characterized in that, Different flashing areas flash at different flashing frequencies; before sending the instruction to be executed determined based on the user's EEG signal to the controlled device, the method further includes: For each of the plurality of instruction targets, a template signal corresponding to the instruction target is determined based on the flashing frequency of the flashing region associated with the instruction target, thereby obtaining a plurality of template signals; Calculate the similarity between the user's EEG signal and each of the template signals to obtain multiple similarities; Based on the multiple similarities, the template signal with the highest similarity to the user's EEG signal among the multiple template signals is determined as the target template signal, and the instruction indicated by the first instruction target corresponding to the target template signal is determined as the instruction to be executed.

4. A signal processing method based on a brain-controlled system, characterized in that, A controlled device applied in a brain control system, the method comprising: The device receives an execution command determined based on the user's EEG signal from a stimulation terminal. The central area of ​​the target interface displayed on the stimulation terminal is used to display a first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and a flashing area associated with each command target. The user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among the multiple command targets. During the execution of the operation corresponding to the instruction to be executed, a second image is acquired in real time; The second image is sent to the stimulation terminal so that the stimulation terminal can determine the execution result of the controlled device performing the operation based on the second image. If the execution result does not meet the preset conditions, the second image is updated to the first image, and the process returns to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation terminal, until the execution result meets the preset conditions. The central area, edge area, and display areas and flashing areas corresponding to multiple instruction targets in the edge area of ​​the target interface are determined by the stimulation end based on the interface layout information of the target interface; the central area, the display areas corresponding to the multiple instruction targets, and the flashing areas do not overlap with each other; The interface layout information includes the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target; The stimulation end determines the target distance between the user's eye position point and the target interface, and calculates the relative position between each instruction target and the flashing area associated with the instruction target based on the target distance. The relative position includes relative angle and relative distance. Based on the relative position between each instruction target and the flashing area associated with the instruction target, the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target are calculated.

5. A brain control system, characterized in that, The brain control system includes: a data acquisition terminal, a stimulation terminal, and a controlled device, wherein the data acquisition terminal is communicatively connected to the stimulation terminal, and the stimulation terminal is communicatively connected to the controlled device; The acquisition end is used to acquire the user's electroencephalogram (EEG) signal when the target interface is displayed on the stimulation end, and to feed back the user's EEG signal to the stimulation end; wherein, the central area of ​​the target interface is used to display a first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and a flashing area associated with each command target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among the multiple command targets; The stimulation end is used to send an execution command determined based on the user's electroencephalogram (EEG) signal to the controlled device. The controlled device is used to acquire a second image in real time and send the second image to the stimulation end during the execution of the operation corresponding to the instruction to be executed; The stimulation end is also used to receive a second image sent by the controlled device; determine the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image, and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops. The stimulation end is also used to acquire interface layout information of the target interface; based on the interface layout information, to determine the central region and edge region of the target interface, and to determine the display area and multiple flashing areas corresponding to multiple instruction targets in the edge region, so that the central region, the display areas corresponding to the multiple instruction targets, and the multiple flashing areas do not overlap; the interface layout information includes the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target; The stimulation end is also used to determine the target distance between the user's eye position point and the target interface; based on the target distance, calculate the relative position between each instruction target and the flashing area associated with the instruction target, the relative position including relative angle and relative distance; based on the relative position between each instruction target and the flashing area associated with the instruction target, calculate the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target.

6. A signal processing device based on a brain-controlled system, characterized in that, A stimulation terminal for use in a brain-controlled system, the device comprising: The acquisition module is used to acquire the user's electroencephalogram (EEG) signal when the target interface is displayed on the stimulation end; the central area of ​​the target interface is used to display a first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple instruction targets and a flashing area associated with each instruction target; the user's EEG signal is generated by flashing induction based on the flashing area associated with the first instruction target when the user is looking at the first instruction target among the multiple instruction targets. The sending module is used to send an execution instruction determined based on the user's EEG signal to the controlled device, so that the controlled device can acquire a second image in real time and send the second image to the stimulation end during the execution of the operation corresponding to the execution instruction; A receiving module is used to receive a second image sent by the controlled device; The determination module is used to determine the execution result of the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image, and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation end, until the execution result meets the preset conditions and stops. The acquisition module is also used to acquire the interface layout information of the target interface; The determining module is further configured to determine the central region and edge region in the target interface based on the interface layout information, and to determine the display area and multiple flashing areas corresponding to multiple instruction targets in the edge region, so that the central region, the display areas corresponding to the multiple instruction targets and the multiple flashing areas do not overlap. The interface layout information includes the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target; The determining module is also used to determine the target distance between the user's eye position point and the target interface; The calculation module is used to calculate the relative position between each instruction target and the flashing area associated with the instruction target based on the target distance, wherein the relative position includes a relative angle and a relative distance; The calculation module is further configured to calculate the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target based on the relative position between each instruction target and the flashing area associated with the instruction target.

7. A signal processing device based on a brain-controlled system, characterized in that, A controlled device used in a brain-computer interface system, the device comprising: The receiving module is used to receive the execution command determined based on the user's EEG signal sent by the stimulation terminal. The central area of ​​the target interface displayed by the stimulation terminal is used to display the first image acquired in real time by the controlled device, and the edge area of ​​the target interface is used to display multiple command targets and a flashing area associated with each command target. The user's EEG signal is generated by flashing induction based on the flashing area associated with the first command target when the user is looking at the first command target among the multiple command targets. The execution module is used to acquire the second image in real time during the execution of the operation corresponding to the instruction to be executed; The sending module is used to send the second image to the stimulation terminal, so that the stimulation terminal can determine the execution result of the controlled device performing the operation based on the second image, and if the execution result does not meet the preset conditions, update the second image to the first image, and return to the step of obtaining the user's EEG signal when the target interface is displayed on the stimulation terminal, until the execution result meets the preset conditions and stops. The central area, edge area, and display areas and flashing areas corresponding to multiple instruction targets in the edge area of ​​the target interface are determined by the stimulation end based on the interface layout information of the target interface; the central area, the display areas corresponding to the multiple instruction targets, and the flashing areas do not overlap with each other; The interface layout information includes the display size of the display area corresponding to the instruction target, the display size of the flashing area, and the relative position between the display area of ​​each instruction target and the flashing area associated with the instruction target; The stimulation end determines the target distance between the user's eye position point and the target interface, and calculates the relative position between each instruction target and the flashing area associated with the instruction target based on the target distance. The relative position includes relative angle and relative distance. Based on the relative position between each instruction target and the flashing area associated with the instruction target, the display size of the display area corresponding to the instruction target and the display size of the flashing area associated with the instruction target are calculated.

8. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the signal processing method based on the brain control system as described in any one of claims 1 to 4.

Images