Brain-computer interaction-based internet of things device control method, system, device and medium

Abstract

The present application relates to a kind of brain-computer interaction-based internet of things equipment control method, system, device and medium.The method comprises: extracting multiple discrete physiological action events from the collected multi-modal bioelectric signal;Maintain a finite state machine containing idle state, pending state, execution state and refractory state;After capturing the first event as target selection event in idle state, it flows to pending state;If a second event different from the first event is captured within a defined time window, enter execution state to lock control intent;Generate a control message based on MQTT protocol and publish to the device control topic, then enter refractory period to suppress repeated triggering;Robustness is also improved by auxiliary scoring and continuous control degradation mechanism.The present application effectively solves the problem of transient action false touch, realizes device decoupling interconnection and closed-loop feedback.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and Internet of Things (IoT) control technology; specifically, this invention relates to a method, system, device, and medium for controlling IoT devices based on brain-computer interaction. Background Technology

[0002] With the development of neuroscience and artificial intelligence, brain-computer interface (BCI) technology has gradually penetrated from the field of medical rehabilitation to the consumer electronics field. In smart home scenarios, using EEG signals to control devices such as lights, curtains, and televisions is not only an important assistive means for people with physical disabilities, but also an important entry point for the next generation of seamless interaction.

[0003] Existing related technical solutions typically focus on improving front-end signal processing and classification algorithms. A typical technical approach involves acquiring multi-channel EEG signals, using neural networks for classification, setting probability thresholds, and directly driving the device when the classification result exceeds the threshold.

[0004] However, existing technologies suffer from significant system-level defects when applied to complex real-life scenarios: First, they lack a temporal-based accidental trigger protection mechanism adapted to transient physiological movements. Existing technologies either employ simple instantaneous probability triggering (extremely prone to accidental activation) or forcibly use "time window integration and accumulation" for judgment. However, for discrete physiological movements such as blinking, lateral eye movements, and teeth clenching, which are inherently transient, forcing users to maintain the movement for "accumulation" judgment violates physiological habits and leads to severe operational delays. Furthermore, some existing improvement solutions use "repeating the same action (such as blinking twice)" as secondary confirmation. This isomorphic action confirmation mechanism still has a very high probability of accidental triggering during natural daily facial activities, resulting in limited effectiveness in preventing accidental activation. Second, the IoT communication architecture is closed, lacking interconnectivity and interoperability, and cannot be connected to modern smart home networks. Third, the interaction process is in an open-loop state, lacking state synchronization, and users cannot be certain whether the device has actually executed the instruction. Summary of the Invention

[0005] In view of this, the present invention provides a method, system, device and medium for controlling Internet of Things devices based on brain-computer interaction, thereby solving or at least alleviating one or more of the above-mentioned problems and other problems existing in the prior art.

[0006] To achieve the aforementioned objective, a first aspect of the present invention provides a method for controlling an Internet of Things (IoT) device based on brain-computer interface, comprising the following steps:

[0007] Multimodal bioelectric signals collected by wearable devices are acquired through the data channel, and multiple discrete physiological action events are extracted and identified as control inputs.

[0008] Maintain a finite state machine, which includes an idle state, a pending state, an execution state, and an inactive state;

[0009] After capturing the first event as the target selection event in the idle state, the finite state machine enters the pending confirmation state with a finite time window;

[0010] The finite state machine enters the execution state and issues a control command only when a specific second event is captured as a confirmation event within the time window of the pending confirmation state. If the event occurs outside the time window, the event fails and the machine automatically resets. After the control command is executed, the finite state machine is forced into a refractory period to suppress repeated triggering.

[0011] After locking the current control intent, a control message is generated based on the MQTT protocol, and the control message is published to the corresponding device control topic. By subscribing to the device status topic, the device status on the local interactive interface is synchronously updated based on the device shadow mechanism.

[0012] Optionally, in the method described above, the method further includes a heterogeneous event confirmation mechanism, in which the second event that triggers the execution state must be heterogeneous with the first event in terms of physiological modality or action characteristics in the pending confirmation state.

[0013] The discrete physiological motor events include: single blink, double blink, leftward lateral eye movement, rightward lateral eye movement, single teeth clenching, and double teeth clenching.

[0014] Optionally, in the method described above, during the detection phase of discrete physiological action events, an auxiliary scoring mechanism is included, the auxiliary scoring mechanism comprising:

[0015] The signal amplitude, energy characteristics, and duration within the time window of the candidate event are extracted and comprehensively scored to calculate the auxiliary confidence level, and substandard noise interference is eliminated.

[0016] Optionally, in the method described above, the method further includes a continuous control degradation mechanism, the continuous control degradation mechanism comprising:

[0017] For continuous control scenarios, the continuous intent features are integrated with energy within a sliding time window, and the energy integration is mapped to the rate of change of the continuous control variables.

[0018] The continuous control scenarios include: long press adjustment, continuous cursor movement, and polling scan rate adjustment.

[0019] In the method described above, optionally, the device control topic adopts a hierarchical structure, which includes user identifier, room identifier, device type, and device identifier;

[0020] The control messages and status messages returned by the physical devices are in JSON format. Their payloads include a header and a body. The header includes a unique message identifier, a timestamp, and an instruction source identifier. The body includes action primitives and extended parameter objects.

[0021] In the method described above, optionally, the synchronous update of the local device state based on the device shadow mechanism includes:

[0022] After the user sends a control command, the corresponding device icon on the interactive interface is set to a waiting state;

[0023] After the communication module detects a message reporting the device status topic, it updates the device shadow data in the local memory.

[0024] The interactive interface monitors changes in the device shadow data, and when the data is updated, the corresponding device icon is refreshed to match the physical device.

[0025] Optionally, in the method described above, the method also supports obtaining control intent through a polling navigation mode, including:

[0026] Render icons for all controllable devices in the interactive interface;

[0027] The cursor can be automatically moved between device icons at a fixed frequency, or the cursor can be moved by controlling the brainwave signal.

[0028] When the cursor moves to the target device, the target device is selected by an EEG signal representing the intention to confirm.

[0029] After the target device is selected, a secondary control floating window pops up on the interactive interface, which includes all executable operations for the target device;

[0030] When the cursor moves to the target operation, the target operation is selected by an EEG signal representing the intention to confirm.

[0031] To achieve the foregoing objectives, a second aspect of the present invention provides a brain-computer interface-based Internet of Things (IoT) device control system, the system being configured to perform the method as described in any one of the first aspects above, the system comprising:

[0032] The sensing access module is used to acquire multimodal bioelectric signals collected by wearable devices from the data channel, and extract and identify multiple discrete physiological action events as control inputs;

[0033] The core logic module is used to determine the control intent and output a definite control event through the confirmation of the second event and the finite state machine transition mechanism.

[0034] The network communication module is used to generate and publish control messages based on the MQTT protocol, and to synchronize the device status on the local interactive interface based on the device shadow mechanism.

[0035] The interaction representation module is used to render the graphical user interface, realize the interaction between the user and the device, and synchronously display the physical device status;

[0036] The perception access module is connected to the core logic module, the core logic module is connected to the network communication module and the interaction representation module, and the network communication module is connected to the interaction representation module.

[0037] To achieve the foregoing objectives, a third aspect of the present invention provides an electronic device including a processor and a memory connected to the processor, the memory storing a computer program executable by the processor, the computer program, when executed by the processor, implementing the method as described in any one of the foregoing first aspects.

[0038] To achieve the foregoing objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any one of the first aspects above.

[0039] The present invention provides an IoT device control method based on brain-computer interface, which effectively solves the problem of accidental touch during transient actions through secondary confirmation events; standardized messages based on the MQTT protocol realize device decoupling and interconnection; and the device shadow mechanism ensures the synchronization of the interactive interface and the physical device status, realizing closed-loop feedback.

[0040] The present invention further provides a brain-computer interface-based Internet of Things (IoT) device control system for performing the above-described methods, as well as an electronic device and a computer-readable storage medium, and thus the system, the electronic device, and the computer-readable storage medium also have the aforementioned advantages. Attached Figure Description

[0041] The disclosure of this invention will become more apparent from the accompanying drawings. It should be understood that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this invention. In the drawings:

[0042] Figure 1 This is a flowchart illustrating an embodiment of the brain-computer interface-based IoT device control method of the present invention.

[0043] Figure 2 This is a schematic diagram of the state transition of a finite state machine in an embodiment of the brain-computer interface-based IoT device control method of the present invention.

[0044] Figure 3This is a schematic block diagram of an embodiment of the Internet of Things (IoT) device control system based on brain-computer interface of the present invention. Detailed Implementation

[0045] Referring to the accompanying drawings and specific embodiments, the structure, composition, features, and advantages of the brain-computer interface-based Internet of Things (IoT) device control method, system, device, and medium of the present invention will be described by way of example below. However, all descriptions should not be construed as limiting the present invention in any way.

[0046] Furthermore, for any single technical feature described or implied in the embodiments mentioned herein, or any single technical feature shown or implied in the various figures, the present invention still allows for any combination or deletion of these technical features (or their equivalents) without any technical obstacle, and thus these further embodiments according to the present invention should also be considered within the scope of this description.

[0047] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0048] It should also be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections, electrical connections, or communication connections; they can refer to wired connections or wireless connections; they can refer to direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0049] One embodiment of the present invention provides a method for controlling Internet of Things (IoT) devices based on brain-computer interface.

[0050] In this embodiment, the method operates on a smart interactive terminal, which may include, but is not limited to, tablet computers, PC (personal computer) central control screens, and computing units. This method can convert non-invasive EEG signals and other bioelectrical signals into standardized IoT control commands and has a system-level false trigger protection mechanism, making it applicable to smart home control.

[0051] like Figure 1 As shown, the method provided in this embodiment includes the following steps.

[0052] Step S1, Sensing Access and Event Extraction: Acquire multimodal bioelectrical signals collected by the wearable device through a data channel. For example, the data channel can be a high-bandwidth data channel established with the wearable EEG acquisition device via Bluetooth or USB (Universal Serial Bus) interface, and the wearable device can be a wearable EEG acquisition device. The system access layer analyzes the signals in real time, extracts and defines multiple types of discrete physiological action events, including but not limited to: eye movement events (such as single blink, double blink, left lateral eye movement, right lateral eye movement) and electromyographic events (such as single teeth clenching, double teeth clenching).

[0053] During the event detection phase, the system does not perform power assessment on transient actions. Instead, it employs an auxiliary confidence scoring mechanism: a comprehensive score is calculated based on the signal amplitude, signal energy, and duration within the candidate event window to determine the auxiliary confidence score. This confidence score is only part of the event recognition algorithm, used to improve the robustness of recognition and eliminate noise interference, rather than being the logical center of the interaction.

[0054] Step S2, Anti-accidental touch control based on timing decision and heterogeneous confirmation: The system maintains a finite state machine containing "idle state, pending confirmation state, execution state, and no-response state". Its state transition logic is as follows:

[0055] Idle State: The system waits for a valid first event input. When a preset "first event" (e.g., a rightward lateral eye movement) is captured, the system interprets it as a target selection event, locks the interaction focus (e.g., selecting "living room light" on the UI), and the state machine transitions to the pending confirmation state.

[0056] Pending Confirmation: The system opens a limited time window T_confirm (e.g., 2000 milliseconds). Within this window, the system waits for the user's confirmation command. To significantly reduce accidental triggering during daily physiological activities, the system implements a heterogeneous event confirmation mechanism, which mandates that the captured "second event" used as the confirmation event must be heterogeneous to the first event in terms of physiological modality or action characteristics (e.g., the first event is lateral eye movement, while the second event is configured as a single teeth clenching). If a qualified heterogeneous second event is captured within T_confirm, the verification passes, and the system jumps to the execution state; if the window times out, or if an action is isomorphic to the first event or an invalid action is captured during this period, the event is automatically reset. Automatic reset includes: the system automatically returning to the idle state and releasing the interactive focus.

[0057] Executing State: The intent is confirmed as a valid instruction, the current control intent is locked, and step S3 is triggered to generate the IoT control instruction. After execution, the state machine automatically transitions to the non-responsive state.

[0058] Refractory State: Forces the system to enter a cooldown period T_cool (e.g., 800 milliseconds), during which the system blocks and clears all new physiological event inputs. It automatically resets to the idle state after the cooldown ends. This state effectively suppresses repeated command triggering caused by twitching of the user's facial muscles.

[0059] One or more embodiments of the present invention also provide an optional implementation: a continuous control degradation mechanism based on sliding window integrals.

[0060] For continuous control scenarios that do not require transient triggering (e.g., polling scan rate adjustment, continuous cursor movement, and long-press continuous color temperature adjustment), a sliding window integration mechanism can be used. In these specific scenarios, the system's decision no longer relies on the aforementioned discrete-time logic, but instead integrates the user's continuous intent characteristics within a sliding time window, directly mapping the integral value to the rate of change of the continuous control variable, thus balancing transient high-frequency response with continuous fine-tuning.

[0061] Step S3, Instruction Generation and Sending: After the finite state machine enters the execution state, it locks the current control intent, generates a control message including action primitives based on the MQTT (Message Queuing Telemetry Transport) protocol, and publishes the control message to the corresponding device control topic.

[0062] This step implements standardized Internet of Things (IoT) communication, enabling the brain-controlled terminal to become a standard IoT control node and decouple it from smart home devices when applied to smart home control. During execution, it does not directly manipulate hardware GPIO (General Purpose Input Output), but instead distributes messages through a publish / subscribe (Pub / Sub) model.

[0063] For example, the network access and protocol stack configuration in this step is as follows: At the physical layer, the WiFi (Wireless Fidelity, a standard wireless LAN) module interface of the terminal operating system is called to support the IEEE 802.11 series standards (a common transmission standard for wireless LANs) and automatically connect to the home wireless router; at the transport layer, a standard TCP / IP (Transmission Control Protocol / Internet Protocol) connection is established to ensure the orderly transmission of data packets; at the application layer, the MQTT client protocol stack is integrated.

[0064] For example, this embodiment designs a hierarchical, semantic topic structure, which supports precise addressing of multiple rooms and multiple device types when applied to smart home control. The device control topic is used by the terminal to send control commands to controllable devices. Its hierarchical structure includes user identifier, room identifier, device type, and device identifier, for example: "home / {user_id} / {room_id} / {device_type} / {device_id} / set", where user_id is used to distinguish different users, room_id is used to distinguish rooms (e.g., "living_room"), device_type is used to distinguish device types (e.g., "light"), and device_id is a unique device identifier. The device status topic is used by the terminal to subscribe to status information reported by physical devices (in the form of status messages). Its structure can utilize wildcards to subscribe to the status reports of all devices in the house, used to monitor device changes in real time and update the interactive interface, for example: "home / {user_id} / + / + / + / status", where "+" is a wildcard.

[0065] For example, all communication messages in this embodiment, including generated control messages and status messages, adopt the lightweight JSON (JavaScript Object Notation) format. The message payload includes a header and a body, possessing scalability and readability. The header of the control message includes a unique message identifier (used to prevent replay attacks and link tracing), a timestamp (millisecond level), and a command source identifier. The body includes action primitives (e.g., "turn_on", "turn_off", "set_level") and extended parameter objects (e.g., brightness value, color temperature, gradient time). The status message includes the device ID, device online status, status objects (e.g., current on / off status, brightness, and other physical attributes), and fault codes.

[0066] In terms of link maintenance and anomaly handling, this embodiment constructs a multi-layered communication assurance mechanism. First, to balance transmission reliability and real-time performance, the system configures different Quality of Service (QoS) levels based on message importance: for critical commands such as switch control, the QoS level is set to 1, meaning "at least once," to ensure the control intent reliably reaches the device; while for high-frequency sensor data (such as real-time power), the QoS level is set to 0, prioritizing real-time data transmission. Second, to maintain the stability of the communication link between the terminal and the server, a heartbeat keep-alive strategy is implemented. The communication module sends an MQTT PingReq (heartbeat request) message to the message broker server every certain interval (e.g., 60 seconds) to continuously maintain the validity of the TCP long connection and prevent connection interruption due to prolonged lack of data interaction. Furthermore, a network reconnection mechanism is designed to address network instability: when a disconnection of the underlying Socket connection is detected, the system automatically enters offline mode, temporarily storing the commands to be sent in a local message queue; once the WiFi connection is detected to be restored, the commands in the queue are automatically retransmitted. Meanwhile, to prevent expired instructions from causing device status chaos, it is supported to set an expiration time (TTL, Time To Live) for instructions in the queue. If an instruction fails to be sent successfully within the expiration time, it will be automatically discarded.

[0067] Step S4, Closed-loop feedback: By subscribing to the device status topic, receive the status messages returned by the physical device, and synchronously update the local device status based on the device shadow mechanism.

[0068] To address the uncertainties introduced by open-loop blind control, this step employs device shadowing technology. A device shadow is a virtual copy of a physical device in software memory. The specific implementation logic includes: First, shadow maintenance: maintaining a JSON object tree in local memory to store the latest status copies of all devices in the house; During the sending phase, an optimistic update strategy is used: when a user sends a control command (e.g., "turn on the light"), the corresponding device icon on the interactive interface enters a "Pending" state, indicating that the command is being transmitted; for example, the icon may display as a rotating loading circle or become semi-transparent; Then, the confirmation phase begins, performing status synchronization operations, continuously monitoring device status topics. Once a status message is received from a physical device, the device shadow data in local memory is immediately updated; for example, upon receiving {"power": If "on" is selected, the switch state is updated to "on". Next is the rendering phase, where the UI layer listens for changes in the device shadow data using the observer pattern. When the data is updated, the interface is immediately refreshed, updating the corresponding device icon to match the physical device's state (e.g., "highlighted real-world state (Active)"), achieving eventual consistency. Additionally, exception handling logic is included. If no device feedback is received after a preset time (e.g., 3000ms), the UI automatically cancels the waiting state and pops up a text message such as "Device not responding," which may be accompanied by vibration feedback, prompting the user to check the network or device power.

[0069] Optionally, depending on the differences in mind control abilities of different users (e.g., the number of induced intention categories) and the number of controlled devices, this embodiment supports two dynamically switchable interaction modes: Direct Mapping Mode and Scanning Navigation Mode.

[0070] The direct mapping mode is suitable for controlling a single device (such as a desk lamp) or a very small number of critical devices. In practice, a lookup table is created to directly map the discrete classification labels (LabelIDs) output by the algorithm to MQTT publishing actions. For example, "left lateral eye movement" is configured to switch to the previous device, and "right lateral eye movement" is configured to switch to the next device (at this point, the target selection phase is underway). Once the target device is selected, based on the heterogeneous confirmation rules, "single bite" is configured to send a state inversion (Toggle) command, and "double bite" is configured to enter the parameter adjustment submenu for that device.

[0071] The polling-based navigation mode is suitable for scenarios where dozens or even hundreds of devices throughout a house can be controlled with a limited number of EEG command categories, typically only 2-3 categories. The specific implementation logic includes matrix rendering, cursor polling, intent capture, and hierarchical control. In the matrix rendering step, the UI renders icons for all controllable devices, for example, as an M x N grid matrix. During cursor polling, two cursor movement methods can be used: automatic or manual. In automatic mode, the highlighted cursor automatically jumps between device icons at a fixed frequency (e.g., 1Hz); in manual mode, the user controls cursor movement via EEG signals, for example, using "left hand imagination" to move the cursor left and "right hand imagination" to move it right. In the intent capture step, when the cursor moves to the target device the user wants to control, the user selects the device by emitting a single EEG signal representing confirmation of intent, such as instantly increasing focus or performing a teeth-clenching motion as confirmation. This mode supports hierarchical control. After selecting a device, a secondary control floating window pops up, which is used to select the operable operation (such as turning the desk lamp on, off, brightening, or dimming). The cursor enters the secondary menu and cycles through it. It also supports the two cursor movement methods mentioned above. When the cursor moves to the target operation that the user wants to perform, the user sends a confirmation to select the operation, and then the specific operation can be executed.

[0072] In both the direct mapping mode and the polling scan navigation mode described above, the aforementioned anti-mistouch decision mechanism can be applied to the process of EEG signal control.

[0073] Based on the same principles as the methods provided in the above embodiments, this invention also provides a brain-computer interface-based Internet of Things (IoT) device control system. This system can be a software system running on an intelligent interactive terminal (including but not limited to tablet computers, PC central control screens, and computing units).

[0074] like Figure 3 As shown, in this embodiment, to achieve modularity and high reliability, the system architecture is strictly divided into four logical levels from bottom to top, corresponding to four core modules:

[0075] The perception access module corresponds to the perception access layer of the system. This module is responsible for hardware abstraction and signal preprocessing. Specifically, it establishes a high-bandwidth data channel with the wearable EEG acquisition device via a means such as Bluetooth or USB interface, acquires multimodal bioelectrical signals collected by the wearable device from the data channel, and extracts and identifies multiple types of discrete physiological action events as control inputs.

[0076] The core logic module, corresponding to the core logic layer of the system, is the decision-making center of the system. This module contains the core anti-accidental touch decision controller, responsible for transforming unstable and noisy probability signals into deterministic control events and maintaining the system's finite state machine. Specifically, this module executes a heterogeneous event confirmation mechanism and maintains the transition logic between idle, pending confirmation, execution, and ineligible states. In the pending confirmation state, this module can also be responsible for integrating the intent features within a sliding time window through a continuous control degradation mechanism, mapping the energy integration to the rate of change of continuous control variables. This module constructs a finite state machine between the algorithm output interface and the command sending interface, avoiding or at least mitigating the problem of accidental touches caused by signal jitter, which is easily caused by directly using the real-time probability output of the classifier as the trigger source in existing technologies.

[0077] The network communication module, corresponding to the system's network communication layer, is responsible for IoT communication and serves as the system's external bus. This module handles WiFi connection management, TCP / IP stack maintenance, MQTT protocol encapsulation, JSON data serialization / deserialization, and a disconnection reconnection keep-alive mechanism. Specifically, this module integrates the MQTT client protocol stack, configures the Topic hierarchy structure, and generates a JSON payload. For link keep-alive, the module periodically (e.g., every 60 seconds) sends an MQTT PingReq message to maintain the TCP long connection; during network disconnection and reconnection, it automatically enters offline mode and stores pending commands in a local message queue.

[0078] The interaction representation module corresponds to the system's interaction representation layer, i.e., the user's human-computer interaction interface. This module is responsible for rendering the graphical user interface (GUI), including a whole-house device matrix view, device status feedback, and multimodal feedback generation. For example, it uses the observer pattern to monitor changes in device shadow data, achieving closed-loop synchronization between UI state and physical device state.

[0079] The IoT device control system based on brain-computer interface provided in this embodiment of the invention can execute the IoT device control method based on brain-computer interface provided in this embodiment of the invention. The implementation principle is similar. The actions performed by each module in the system of each embodiment of the invention correspond to the steps in the method of each embodiment of the application. For detailed functional descriptions of each module, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0080] This invention also provides an electronic device, which may include a memory and a processor connected to the memory. The memory may store a computer program, and the processor may be used to execute the computer program in the memory to implement the steps of the brain-computer interface-based Internet of Things device control method as described in any of the foregoing embodiments.

[0081] This invention also provides a computer-readable storage medium on which a computer program can be stored, wherein the computer program, when executed, implements the steps of the brain-computer interface-based Internet of Things device control method as described in any of the foregoing embodiments.

[0082] It should be noted that the computer-readable storage medium described above in this invention can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0083] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0084] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic device and computer-readable storage medium described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0085] To verify the effectiveness and advancement of the embodiments of the present invention, a fully functional simulated home experimental environment was constructed as an example:

[0086] At the hardware level, a high-performance tablet computer is used as the control terminal, and the system described in the above embodiments is installed. The EEG acquisition device is a 24-channel wireless EEG cap with a sampling rate set to 250Hz, establishing a data connection with the control terminal via Bluetooth. For the network environment, a WiFi 6 router is deployed, and the open-source Mosquitto MQTTBroker is deployed as a message broker server within the local area network. The controlled device group includes: a smart ceiling light supporting WiFi connectivity (with brightness and color temperature adjustment functions), an electric curtain connected via a Zigbee (a wireless communication technology) gateway, and a smart fan connected via an infrared repeater.

[0087] Based on the above environment, experiments were conducted to test the control efficiency of multiple devices throughout the house under uncontrolled conditions.

[0088] The purpose of this test was to verify the practicality of the aforementioned polling and scanning navigation mode combined with MQTT communication. During the test, the subjects were required to complete a complex sequence of instructions, specifically: "Select the desk lamp → Turn it on → Adjust the brightness to 50% → Select the curtains → Turn them off".

[0089] Test results showed that in polling scan navigation mode, subjects could complete the entire operation in an average of 22 seconds using a single teeth-gripping confirmation signal, with a success rate of 98%. Meanwhile, the average end-to-end latency from intent confirmation to device action was measured to be 180ms (WiFi network latency), and after each operation, the software interface icons were able to correctly synchronize with the device's actual physical state within 250ms.

[0090] This test verifies that the system of this embodiment has extremely high availability, low communication latency, and a complete interactive feedback loop, which can meet the daily home control needs in non-medical scenarios.

[0091] In summary, the embodiments of the present invention may have, or partially have, the following beneficial effects:

[0092] (1) To address the problem of high false triggering rate of transient physiological actions in existing brain-controlled smart home systems under complex living environments and the inapplicability of existing power-saving mechanisms, this invention provides a highly robust system-level anti-false triggering safety mechanism. This invention abandons the single logic of forcibly powering transient actions and innovatively proposes a time-series decision-based anti-false triggering mechanism based on "first event selection + heterogeneous second event confirmation within a time window". By separating the target selection and instruction confirmation stages through a finite state machine, and utilizing the combination confirmation of different physiological channels (such as electrooculography and electromyography), the accidental false triggering caused by daily natural physiological activities (such as continuous blinking) is greatly reduced; and the refractory period after execution shields against repeated triggering caused by muscle twitching. At the same time, this invention uses sliding window integral degradation as an optional implementation method for auxiliary confidence calculation and continuous control, so that the system has both transient high-frequency response capability and can handle continuous fine operations such as long-press adjustment.

[0093] (2) To address the issue of non-standard communication protocols preventing access to mainstream IoT ecosystems, a standardized IoT full-connectivity architecture based on MQTT was constructed. This system integrates an MQTT communication protocol stack. By defining a standardized Topic hierarchy and JSON Payload specification, valid intentions that have passed the anti-misclick judgment are mapped into control messages containing unique message IDs, timestamps, and action primitives, achieving complete decoupling between mind-controlled intentions and physical devices. As long as this protocol is followed, any third-party device supporting WiFi / Zigbee gateways can be controlled.

[0094] (3) To address the lack of closed-loop status feedback in human-computer interaction, a digital twin closed-loop interaction mechanism based on device shadows is introduced. The system is equipped with a status subscriber that subscribes to device status topics and parses the status messages returned by the physical devices. This ensures that the status updates of the terminal software UI are no longer directly driven by user operations, but by the status messages returned by the physical devices. This ensures that the on / off and brightness states displayed on the software interface remain consistent with the physical world, achieving true closed-loop interaction.

[0095] (4) A polling-based device navigation control mode based on a single instruction stream is provided. The device matrix polling function is provided in the interactive interface. The controllable devices are traversed by an automatic timed jump or a manually controlled cursor movement mechanism. Users can use only a single type of EEG intention signal (as a confirmation key). With the optional sliding window integral decision mechanism, hierarchical control of the selection of a large number of IoT devices, entry into secondary menus and parameter adjustment can be realized.

[0096] The technical scope of this invention is not limited to the contents of the above specification. Those skilled in the art can make various modifications and variations to the above embodiments without departing from the technical concept of this invention, and all such modifications and variations should fall within the scope of this invention.

Claims

1. A method for controlling Internet of Things (IoT) devices based on brain-computer interface, characterized in that, Includes the following steps: Multimodal bioelectric signals collected by wearable devices are acquired through the data channel, and multiple discrete physiological action events are extracted and identified as control inputs. Maintain a finite state machine, which includes an idle state, a pending state, an execution state, and an inactive state; After capturing the first event as the target selection event in the idle state, the finite state machine enters the pending confirmation state with a finite time window; The finite state machine enters the execution state and issues control instructions only when a specific second event is captured as a confirmation event within the time window of the pending confirmation state. If it occurs outside the time window, the event fails and the machine is automatically reset. After the control command is executed, the finite state machine is forced into a refractory period to suppress repeated triggering. After locking the current control intent, a control message is generated based on the MQTT protocol, and the control message is published to the corresponding device control topic. By subscribing to the device status topic, the device status on the local interactive interface is synchronously updated based on the device shadow mechanism.

2. The method of claim 1, wherein, The method also includes a heterogeneous event confirmation mechanism, which requires that the second event that triggers the execution state must be heterogeneous with the first event in terms of physiological modality or action characteristics in the pending confirmation state. The discrete physiological motor events include: single blink, double blink, leftward lateral eye movement, rightward lateral eye movement, single teeth clenching, and double teeth clenching.

3. The method of claim 1, wherein, In the detection phase of discrete physiological action events, an auxiliary scoring mechanism is included, which includes: The signal amplitude, energy characteristics, and duration within the time window of the candidate event are extracted and comprehensively scored to calculate the auxiliary confidence level, and substandard noise interference is eliminated.

4. The method of claim 1, wherein, The method further includes a continuous control degradation mechanism, which comprises: For continuous control scenarios, the continuous intent features are integrated with energy within a sliding time window, and the energy integration is mapped to the rate of change of the continuous control variables. The continuous control scenarios include: long press adjustment, continuous cursor movement, and polling scan rate adjustment.

5. The method of claim 1, wherein, The device control topic adopts a hierarchical structure, which includes user identifier, room identifier, device type, and device identifier. The control messages and status messages returned by the physical devices are in JSON format. Their payloads include a header and a body. The header includes a unique message identifier, a timestamp, and an instruction source identifier. The body includes action primitives and extended parameter objects.

6. The method of claim 1, wherein, The local device status synchronization update based on the device shadow mechanism includes: After the user sends a control command, the corresponding device icon on the interactive interface is set to a waiting state; After the communication module detects a message reporting the device status topic, it updates the device shadow data in the local memory. The interactive interface monitors changes in the device shadow data, and when the data is updated, the corresponding device icon is refreshed to match the physical device.

7. The method of claim 1, wherein, The method also supports obtaining control intent through polling navigation mode, including: Render icons for all controllable devices in the interactive interface; The cursor can be automatically moved between device icons at a fixed frequency, or the cursor can be moved by controlling the brainwave signal. When the cursor moves to the target device, the target device is selected by an EEG signal representing the intention to confirm. After the target device is selected, a secondary control floating window pops up on the interactive interface, which includes all executable operations for the target device; When the cursor moves to the target operation, the target operation is selected by an EEG signal representing the intention to confirm. 8.A brain-computer interaction based Internet of Things device control system, characterized in that, The system is used to perform the method as described in any one of claims 1-7, the system comprising: The sensing access module is used to acquire multimodal bioelectric signals collected by wearable devices from the data channel, and extract and identify multiple discrete physiological action events as control inputs; The core logic module is used to determine the control intent and output a definite control event through the confirmation of the second event and the finite state machine transition mechanism. The network communication module is used to generate and publish control messages based on the MQTT protocol, and to synchronize the device status on the local interactive interface based on the device shadow mechanism. The interaction representation module is used to render the graphical user interface, realize the interaction between the user and the device, and synchronously display the physical device status; The perception access module is connected to the core logic module, the core logic module is connected to the network communication module and the interaction representation module, and the network communication module is connected to the interaction representation module.

9. An electronic device, comprising: It includes a processor and a memory connected to the processor, the memory storing a computer program executable by the processor, the computer program implementing the method as described in any one of claims 1-7 when executed by the processor.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the method as described in any one of claims 1-7.

Images