Intelligent environment equipment control method and device based on wearable brain-computer interface equipment

Abstract

The invention discloses an intelligent environment equipment control method, device and equipment based on wearable brain-computer interface equipment, and the method comprises the steps: collecting the physiological index data of a specific region of the body surface of a user through the wearable brain-computer interface equipment, and transmitting the physiological index data to a data processing terminal in a Best Efort mode, the physiological index data comprises electroencephalogram data, photoelectric blood flow mapping data, body temperature data and motion data; performing data processing on the physiological index data through the data processing terminal according to a preset decision function to obtain decision data; data stream matrixes corresponding to different sending frequencies are constructed for the decision data, a data packet obtained after the data stream matrixes are packaged is transmitted to corresponding target equipment through an OSC or an LSL or a user-defined upper-layer application protocol, and the target equipment comprises intelligent environment equipment; and performing a corresponding decision operation through the intelligent environment equipment according to the received data packet. Linkage control of the wearable brain-computer interface device and the smart home system can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of wearable brain-computer interface devices, and in particular to a method and device for controlling intelligent environmental equipment based on a wearable brain-computer interface device. Background Art

[0002] In recent years, with the rapid development of brain-computer interface (BCI) technology, controlling external devices by reading and analyzing electrical signals generated by the brain has become a reality. BCI technology is not limited to scientific research; it also demonstrates enormous potential for application in a variety of fields, including healthcare, entertainment, and smart homes. In particular, in the smart home and outpatient healthcare sectors, using wearable BCI devices to collect EEG data and combine it with other physiological indicators to achieve intelligent control of the home environment has become a cutting-edge and challenging research direction.

[0003] However, existing technologies still face numerous challenges in achieving wearable physiological electrical signal acquisition, particularly in linking brain-computer interface devices with intelligent control, particularly smart home systems. On the one hand, the collection, transmission, and processing of physiological indicator data require efficient and stable technical support to ensure data accuracy and real-time performance. On the other hand, smart home devices are diverse, with varying control methods and interface standards. Therefore, achieving seamless connectivity and intelligent control between wearable brain-computer interface devices and smart environment devices remains a pressing issue. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to propose a method, device and equipment for controlling intelligent environmental equipment based on a wearable brain-computer interface device, aiming to solve the above problems.

[0005] To achieve the above objectives, the present invention provides a method for controlling an intelligent environment device based on a wearable brain-computer interface device, the method comprising:

[0006] The wearable brain-computer interface device collects physiological indicator data of a specific area of the user's body surface, and transmits the physiological indicator data to a data processing terminal in Best Effort mode. The physiological indicator data includes EEG data, photoelectric blood flow mapping data, body temperature data, and motion data.

[0007] Processing the physiological indicator data according to a preset decision function by the data processing terminal to obtain decision data;

[0008] Constructing a data flow matrix corresponding to different transmission frequencies for the decision data, and transmitting the data packets encapsulated by the data flow matrix to the corresponding target device through OSC, LSL or a custom upper layer application protocol, wherein the target device includes an intelligent environment device;

[0009] The intelligent environment device performs corresponding decision operations according to the received data packet.

[0010] Preferably, the data processing terminal processes the physiological indicator data according to a preset decision function to obtain decision data, including:

[0011] The data processing terminal performs one or more of time window screening, filtering, time-frequency domain transformation, principal component analysis, and feature extraction on the physiological indicator data to obtain application matrix data;

[0012] The physiological indicator data or the application matrix data is processed by the decision function at a preset frequency, and the obtained high-order data is used as the decision data; wherein the processing of the decision function includes denoising, filtering and linear / nonlinear mapping in the time domain, frequency domain, spatial domain and code domain.

[0013] Preferably, the method of transmitting the data packet encapsulated by the data flow matrix to the corresponding target device through OSC, LSL or a custom upper layer application protocol includes:

[0014] When it is determined that the amplitude or frequency corresponding to the data stream matrix does not match the target device, frequency processing or amplitude processing is performed on the corresponding data stream in the data stream matrix through the data processing terminal or the cloud server; wherein, frequency processing includes performing an up-conversion operation or a down-conversion operation on the frequency of the corresponding data stream in the data stream matrix; amplitude processing includes performing a recalibration operation on the amplitude of the corresponding data stream in the data stream matrix to achieve a target curve within a preset range.

[0015] Preferably, the target device includes a cloud server; after the data packet encapsulated by the data flow matrix is transmitted to the corresponding target device through OSC, LSL or a custom upper layer application protocol, the method includes:

[0016] The cloud server performs operations including authentication, storage, processing and distribution according to the received data packet, and sends the data packet after the distribution operation to the data processing terminal or the indirect controlled device in the intelligent environment device.

[0017] Preferably, the method further comprises:

[0018] When the local area network formed by the data processing terminal and the smart environment device requires a data stream synchronization service, the smart environment device with timestamp synchronization capability in the local area network is selected as a clock source reference, or the smart environment device with the most accurate clock characteristics is selected as a clock source reference through M2M communication between the smart environment devices;

[0019] The data processing terminal performs correction processing on the real-time quantity of the clock relative to the reference based on the clock source reference.

[0020] Preferably, the correction processing of the real-time clock quantity relative to the reference based on the clock source by the data processing terminal includes:

[0021] Adjust the clock by calculating the calibration time; wherein the calibration time t = (T2-T1) + d / 2, d = T4-T1;

[0022] Where T1 is the timestamp when the client sends the NTP request, T2 is the timestamp when the server receives the NTP request, T3 is the timestamp when the server replies to the NTP request, T4 is the timestamp when the client receives the NTP reply, d1 is the transmission delay of the NTP request packet, and d2 is the transmission delay of the NTP reply packet.

[0023] Preferably, after the smart environment device performs a corresponding decision operation according to the received data packet, the method further includes:

[0024] The data processing terminal generates a data visualization chart based on the heartbeat data packet sent by the smart environment device, and the data visualization chart is used to display the fluctuation of real-time physiological indicator data.

[0025] Preferably, the wearable brain-computer interface device includes a brain-computer interface device; the data processing terminal includes a computer, mobile phone, tablet, or gateway device with digital signal processing capabilities; the intelligent environment device includes an actuator with sound, light, electricity, magnetism, chemistry, or mechanical motion response capabilities, or a terminal with data processing and display capabilities.

[0026] To achieve the above objectives, the present invention further provides an intelligent environmental device control device based on a wearable brain-computer interface device, the device comprising:

[0027] A data acquisition unit, configured to collect physiological indicator data from a specific area of the user's body surface through a wearable brain-computer interface device, and transmit the physiological indicator data to a data processing terminal in Best Effort mode. The physiological indicator data includes EEG data, photoelectric blood flow mapping data, body temperature data, and motion data;

[0028] a data processing unit, configured to process the physiological indicator data according to a preset decision function through the data processing terminal to obtain decision data;

[0029] A data stream construction unit is configured to construct a data stream matrix corresponding to different transmission frequencies for the decision data, and transmit the data packets encapsulated by the data stream matrix to the corresponding target device via OSC, LSL or a custom upper layer application protocol, wherein the target device includes an intelligent environment device;

[0030] An execution unit is configured to perform a corresponding decision operation according to the received data packet through the intelligent environment device.

[0031] In order to achieve the above-mentioned objectives, the present invention also proposes an intelligent environment device control device based on a wearable brain-computer interface device, comprising a processor, a memory, and a computer program stored in the memory. The computer program is executed by the processor to implement the steps of an intelligent environment device control method based on a wearable brain-computer interface device as described in the above embodiment.

[0032] In order to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a computer program is stored. The computer program is executed by a processor to implement the steps of a smart environment device control method based on a wearable brain-computer interface device as described in the above embodiment.

[0033] In order to achieve the above-mentioned objectives, the present invention also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a smart environment device control method based on a wearable brain-computer interface device as described in the above embodiment.

[0034] Beneficial effects:

[0035] The above solution uses a wearable brain-computer interface device to collect a variety of physiological indicators, including EEG data, photoelectric blood flow mapping data, body temperature data, and motion data, achieving comprehensive monitoring of the user's physiological state. This data is transmitted to a data processing terminal and processed through a preset decision function to generate decision data for controlling smart environmental devices. This not only improves the intelligence level of the smart home system, but also enhances the system's responsiveness and personalized service capabilities, realizing the coordinated control of the wearable brain-computer interface device and the smart home system.

[0036] The above solution uses a data processing terminal to perform time window screening, filtering, time-frequency domain transformation, principal component analysis, and feature extraction on physiological indicator data, effectively extracting key information and reducing data redundancy. Further processing through a decision function outputs high-order data as a basis for decision-making, improving control accuracy and stability. This enables the system to quickly respond to physiological changes in the user and achieve adaptive adjustments to the intelligent environment.

[0037] The above solution considers the matching between the data flow matrix and the target device, ensuring data compatibility through frequency and amplitude processing. It also uses OSC, LSL, or custom upper-layer application protocols to encapsulate data packets, improving data transmission efficiency and reliability. This flexible data adaptation and protocol encapsulation mechanism enables the system to support a variety of intelligent environmental devices, enhancing system scalability and compatibility.

[0038] The above solution, by introducing cloud servers to process data packets, not only improves data processing capabilities but also enhances data security. The cloud servers can perform authentication, storage, and processing operations to ensure the legitimacy and integrity of the data. Furthermore, the cloud servers can distribute processed data packets to data processing terminals or indirectly controlled devices, enabling real-time data sharing and intelligent control.

[0039] The above solution ensures real-time and accurate data processing by determining whether the LAN requires data stream synchronization services and selecting an appropriate clock source reference for clock correction. This clock synchronization mechanism is crucial for improving system performance and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 A flowchart of a method for controlling an intelligent environmental device based on a wearable brain-computer interface device is provided in accordance with an embodiment of the present invention.

[0042] Figure 2 A schematic diagram of the overall framework of a control system provided by one embodiment of the present invention.

[0043] Figure 3 A schematic structural diagram of a data processing terminal provided by one embodiment of the present invention.

[0044] Figure 4A schematic structural diagram of an intelligent environmental device control device based on a wearable brain-computer interface device provided in one embodiment of the present invention.

[0045] The realization of the objectives of the invention, the functional features and advantages will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention for which protection is sought, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] The present invention is described in detail below with reference to the embodiments.

[0048] Reference Figure 1 The figure is a flow chart of a method for controlling an intelligent environment device based on a wearable brain-computer interface device provided by one embodiment of the present invention.

[0049] In this embodiment, the method includes:

[0050] S11, collecting physiological indicator data of a specific area of the user's body surface through a wearable brain-computer interface device, and transmitting the physiological indicator data to a data processing terminal in Best Effort mode, the physiological indicator data including EEG data, photoelectric blood flow mapping data, body temperature data and motion data.

[0051] In this embodiment, the method is implemented based on the control system, referring to Figure 2As shown, the control system includes a wearable brain-computer interface device, a data processing terminal, a cloud server, and an intelligent environment device. Based on the data network of the control system, the control and feedback loop of various intelligent environment devices are performed by executing the original data such as brain wave electrode data, motion data (derivative data such as angular velocity and its variation), optical signals, body temperature data, battery data, etc. based on one or more channels and the quality data of each sensor signal derived and calculated in the data processing terminal (such as electrode signal quality index, PPG signal quality index, heartbeat signal index, EEG artifacts) as parameters. The wearable brain-computer interface device is a wearable multimodal brain-computer interface device, referring to Figure 3 As shown, a wearable brain-computer interface device collects physiological indicator data from specific areas of the user's body surface. Specifically, the wearable brain-computer interface device collects electrical signals from specific areas of the body surface through flexible electrodes, and collects optical signals from the wearer or accessories (such as photoplethysmography: Photoplethysmography, abbreviated as PPG; fNIRS: high-density near-infrared functional brain imaging system) to collect photoelectric blood flow mapping data at different depths, body temperature data, and motion data such as the angle / speed / acceleration of the motion sensor, thereby forming a comprehensive multi-physiological indicator data of the wearer's current brain electrical data. The above-mentioned physiological indicator data is sent to the data buffer area for storage according to the preset FIFO transmission format, and is wirelessly transmitted in BestEffort mode. It is sent to the data processing terminal directly or through a wireless network transmission at different data transmission frequencies.

[0052] S12, processing the physiological indicator data according to a preset decision function by the data processing terminal to obtain decision data.

[0053] Furthermore, in step S12, the data processing terminal processes the physiological indicator data according to a preset decision function to obtain decision data, including:

[0054] S12-1, the data processing terminal performs one or more of time window screening, filtering, time-frequency domain transformation, principal component analysis, and feature extraction on the physiological indicator data to obtain application matrix data;

[0055] S12-2, processing the physiological indicator data or the application matrix data through the decision function at a preset frequency, and using the obtained high-order data as the decision data; wherein, the processing of the decision function includes denoising, filtering and linear / nonlinear mapping in the time domain, frequency domain, spatial domain and code domain.

[0056] S13, constructing a data flow matrix corresponding to different sending frequencies for the decision data, and transmitting the data packet encapsulated by the data flow matrix to the corresponding target device through OSC, LSL or a custom upper layer application protocol, and the target device includes an intelligent environment device.

[0057] In this embodiment, the data processing terminal refers to a device including a computer, mobile phone, tablet, gateway, etc. with digital signal processing capabilities. Each data processing terminal can connect to one or more brain-computer interface devices (BCI) via a wireless network. x , x=1,2,3...). The data processing terminal directly applies the combination of the biological indicator data or further processes the data before cross-level comprehensive application. Among them, data processing includes time window screening of the biological indicator data (for example, n samples are the window size, each time the sliding window is performed according to the newly entered m samples online, and the overlapping sliding window algorithm or the use of timed composite settlement), filtering (for example, mean, median filtering, subtraction of the time window, etc.) The application matrix data is obtained by performing one or more of the following operations: removing the DC component, or using a Kalman low-pass sliding filter, time-frequency domain transformation, principal component or independent component analysis (such as ICA or PCA), feature extraction (such as information entropy feature, mean feature, correlation coefficient and other feature judgment methods). The biological indicator data or the application matrix data after data processing are respectively divided into a certain frequency (such as EEG data f eeg =256Hz, such as interactive experience data of sound and light control signals f UX =1 / 12 / 24 / 60Hz, such as motion data from accelerometers, PPG sensors, and other sensors sensors =50Hz) is fed into the decision function for data matrix feature determination and online classification and clustering, outputting decision data. The data processed by decision functions in the time, frequency, spatial, and code domains, including denoising, filtering, and linear / nonlinear mapping, is high-order data. The decision function can use three continuous windows corresponding to multiple thresholds to determine postural stability and EEG stability; it can also use the peak frequency height and width of different bands to determine cognitive ability and emotion.

[0058] The data processing terminal uses a certain frequency corresponding to the decision data (such as audio control data User interface update data ) sends a data set (DataSet, DS) to the data receiving port of the execution mechanism software module or peripheral device that is negotiated by the real-time application layer protocol, matched manually / default or matched by the recommended algorithm. y ), that is, the data flow matrix, the different data flows in the data group With different sending frequencies Among them, the execution mechanism software module can execute changes such as sound, vision, machinery, and smell through the interactive modules in the data processing terminal, that is, the execution mechanism software module can equip the data processing terminal with an execution mechanism such as UI; the data transceiver port of the peripheral device sends relevant data to one or more specified network addresses and their corresponding ports; the network address can be the address of one or more cloud servers, or it can be the directly controlled device (DD) in one or more intelligent environment devices. i , i=1,2,3...).

[0059] The data group is used as application layer data to communicate with the controlled device or cloud server in one-way or two-way communication at the application layer through a wireless network or a wired network. The data flow matrix is encapsulated and transmitted based on OSC, LSL or other custom upper-layer application protocols. During the transmission process, control can be achieved by forming a data packet containing the data mode + data type + parameter value of each dimension of data, or creating several data streams with unique identifiers at the application layer and sending the application layer data packet containing the sampling rate, timestamp, number of channels and other information of the hardware device (such as data processing terminal, smart environment device) through multi-threaded sending and event-driven form as data stream publishing and subscription to the receiving end in the network (such as smart environment device). The transmission of the designated port in the IP network based on the UDP or TCP transport layer protocol is read through the designated port by the program in the data processing terminal and then applied.

[0060] Furthermore, in step S13, the data packet encapsulated by the data flow matrix is transmitted to the corresponding target device through OSC, LSL or a custom upper layer application protocol, including:

[0061] When it is determined that the amplitude or frequency corresponding to the data stream matrix does not match the target device, frequency processing or amplitude processing is performed on the corresponding data stream in the data stream matrix through the data processing terminal or the cloud server; wherein, frequency processing includes performing an up-conversion operation or a down-conversion operation on the frequency of the corresponding data stream in the data stream matrix; amplitude processing includes performing a recalibration operation on the amplitude of the corresponding data stream in the data stream matrix to achieve a target curve within a preset range.

[0062] Furthermore, the upsampling operation includes linear interpolation, nearest neighbor interpolation, polynomial interpolation, spline interpolation or Fourier interpolation using upsampling; the downsampling operation includes downsampling decimation, filtering decimation, resampling, post-transformation sampling or compressed sensing; the recalibration operation includes Min-Max normalization, Z-Score normalization, scaling normalization, natural logarithm transformation and square root transformation, Box-Cox transformation, fractional scaling normalization or Sigmoid function mapping.

[0063] In this embodiment, when the amplitude or frequency of the data required by the application corresponding to the smart environment device is different from the received data group (data flow matrix) due to large individual differences, the data flow corresponding to the data flow matrix is frequency processed or amplitude processed through the data processing terminal or cloud server. For frequency processing, methods such as upsampling linear interpolation, nearest neighbor interpolation, polynomial interpolation, spline interpolation, and Fourier interpolation are primarily used to increase the data frequency. Alternatively, methods such as downsampling, filtering, resampling, post-transformation sampling, or compressed sensing are used to reduce the frequency of the data to match the desired effect in the actual environment. For example, in sleep scenarios, the frequency needs to be gradually reduced according to the user's sleep stage, while in game control scenarios, the frequency needs to be increased according to operational requirements. For amplitude processing, when the amplitude at a specific location in the data set differs from the preset limit due to individual EEG changes, the data is recalibrated to the target curve within a specified range through methods such as Min-Max normalization, Z-Score (mean-variance) normalization, scaling normalization, natural logarithm transformation and square root transformation, Box-Cox transformation, decimal scaling normalization, and Sigmoid function mapping. The frequency and amplitude connection of the data stream between the data processing terminal and the intelligent environment device can be preset, manually adjusted during operation, or dynamically adjusted according to the scenario through the above-mentioned frequency or amplitude processing.

[0064] Furthermore, the target device includes a cloud server; after transmitting the data packet encapsulated by the data flow matrix through OSC, LSL or a custom upper layer application protocol to the corresponding target device, the method includes:

[0065] The cloud server performs operations including authentication, storage, processing and distribution according to the received data packet, and sends the data packet after the distribution operation to the data processing terminal or the indirect controlled device in the intelligent environment device.

[0066] In this embodiment, if the cloud server receives the relevant data packet, it can authenticate, store, process and distribute the relevant data packet. The data packet recipient of the distribution operation can be a data processing terminal or an intelligent environment device (such as an indirect controlled device IND j ,j=1,2,3...) Intelligent environment devices include directly controlled devices and indirectly controlled devices. j ,j=1,2,3...) or indirectly controlled device (IND j, j = 1, 2, 3...) can be actuators with response capabilities such as sound, light, electricity (transcranial electrical stimulation, electromagnetic heating, etc.), magnetism, chemistry (release of molecules such as odors), and mechanical motion (motor drive). It can also be various terminals with data processing and display capabilities, such as smart imaging, smart lamps, air conditioning electrodes, magnetic control equipment, robots, programmable light strips and panels, speakers, robotic arms, fragrance machines, and other smart environmental devices.

[0067] Furthermore, the method further comprises:

[0068] When the local area network formed by the data processing terminal and the smart environment device requires data stream synchronization service, the smart environment device with timestamp synchronization capability in the local area network is selected as the clock source reference, or the smart environment device with the most accurate clock characteristics is selected as the clock source reference through M2M communication between the smart environment devices; wherein the smart environment device with the clock source reference serves as the server in the time synchronization network, and other devices that request to initiate the synchronization protocol from the server serve as clients;

[0069] The data processing terminal performs correction processing on the real-time quantity of the clock relative to the reference based on the clock source reference.

[0070] Furthermore, the correction processing of the real-time quantity of the clock relative to the reference based on the clock source reference by the data processing terminal includes:

[0071] Adjust the clock by calculating the calibration time of the difference between the clock source reference and the local device; where the calibration time t = (T2-T1) + d / 2, d = T4-T1;

[0072] Where T1 is the timestamp when the client sends the NTP request, T2 is the timestamp when the server receives the NTP request, T3 is the timestamp when the server replies to the NTP request, T4 is the timestamp when the client receives the NTP reply, d1 is the transmission delay of the NTP request packet, and d2 is the transmission delay of the NTP reply packet.

[0073] In this embodiment, if the local area network formed by the data processing terminal and the smart environment device needs to publish data packets in the form of data stream synchronization timestamps, the user can select a specific device with timestamp synchronization capability in the network as the timestamp reference (i.e., as the clock source reference), or select the smart environment device with the most accurate clock characteristics as the clock source reference through M2M communication between devices. The data processing terminal exchanges UDP data packets n times based on the clock, measures the difference between the clock offset of the devices in each system and the clock source reference, and compares the offset of each data stream with the clock source reference and other data stream real-time quantities based on multiple round trip times (RTT). The time offset between the packet header timestamps of different devices is deducted, and the calibration parameter Δt=t of the NTP protocol is calculated. server -t client , and further use RTT for continuous rolling calibration d = T4 - T1. Assuming d1 = d2, t can also be expressed as: The client adjusts the clock using t or further adjusts the ppm (parts per million) error. Where: T1: timestamp of the client sending the NTP request, T2: timestamp of the server receiving the NTP request, T3: timestamp of the server replying to the NTP request, T4: timestamp of the client receiving the NTP reply, d1: transmission delay of the NTP request packet, d2: transmission delay of the NTP reply packet. Data stream time synchronization calibration and correction are performed based on the offset between the client's local clock value and the server's calibration value, including de-jittering, such as time smoothing, to ensure data synchronization and real-time performance. If the user cannot confirm the clock source reference, the data processing terminal can initiate and periodically initiate clock source stability calibration for multiple data packets, using data information such as minimum RTT jitter, minimum average RTT, and number of network node hops.

[0074] S14: performing corresponding decision-making operations according to the received data packet via the intelligent environment device.

[0075] Furthermore, after the smart environment device performs a corresponding decision operation according to the received data packet, the method further includes:

[0076] S15, the data processing terminal generates a data visualization chart according to the heartbeat data packet sent by the smart environment device, and the data visualization chart is used to display the fluctuation of real-time physiological indicator data.

[0077] In this embodiment, when the smart environment device receives a data packet, it directly executes a predetermined operation or executes a certain corresponding logic or algorithm based on the content of the data packet. This may include but is not limited to determining the clock source within the network, parsing the received and sent data, and the actuator performing a predetermined response based on the data (the actuator's own action, such as arm movement or motor rotation). If the data processing terminal receives a heartbeat, feedback, or data packet containing specific control content from the smart environment device, it performs the following operations based on the data packet feedback content: forming a data visualization chart to visualize the fluctuation of specific physiological indicators (including EEG of each channel, the same below) in real time; deriving the data change interactive interface based on the received feedback or content data or directly using it as input, and changing the interface and interactive elements such as sound, light, and vibration; and storing data for forming a visualization report of the completed project.

[0078] Reference Figure 4 FIG2 is a schematic structural diagram of an intelligent environment device control device based on a wearable brain-computer interface device provided by one embodiment of the present invention.

[0079] In this embodiment, the device 20 includes:

[0080] A data acquisition unit 21 is configured to collect physiological indicator data from a specific area of the user's body surface through a wearable brain-computer interface device and transmit the physiological indicator data to a data processing terminal in a best effort mode. The physiological indicator data includes EEG data, photoelectric blood flow mapping data, body temperature data, and motion data.

[0081] A data processing unit 22 is configured to process the physiological indicator data according to a preset decision function through the data processing terminal to obtain decision data;

[0082] A data stream construction unit 23 is configured to construct a data stream matrix corresponding to different transmission frequencies for the decision data, and transmit the data packets encapsulated by the data stream matrix to the corresponding target device via OSC, LSL or a custom upper layer application protocol, wherein the target device includes an intelligent environment device;

[0083] The execution unit 24 is configured to perform a corresponding decision operation according to the received data packet through the intelligent environment device.

[0084] Furthermore, the device 20 further includes:

[0085] The data stream synchronization unit is used to select the smart environment device with timestamp synchronization capability in the local area network as the clock source reference when the local area network formed by the data processing terminal and the smart environment device requires data stream synchronization service, or select the smart environment device with the most accurate clock characteristics as the clock source reference through M2M communication between the smart environment devices; and perform correction processing on the real-time quantity of the clock relative to the reference based on the clock source reference by the data processing terminal.

[0086] Furthermore, the device 20 further includes:

[0087] A chart display unit is used for the data processing terminal to generate a data visualization chart based on the heartbeat data packet sent by the intelligent environment device, and the data visualization chart is used to display the fluctuation of real-time physiological indicator data.

[0088] Each unit module of the device 20 can respectively execute the corresponding steps in the above method embodiment, so each unit module will not be described in detail here. Please refer to the description of the corresponding steps above for details.

[0089] The embodiment of the present invention further provides an intelligent environment device control device based on a wearable brain-computer interface device, the device comprising the intelligent environment device control device based on the wearable brain-computer interface device as described above, wherein the intelligent environment device control device based on the wearable brain-computer interface device can adopt Figure 4 The structure of the embodiment can be executed accordingly. Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects. For details, please refer to the relevant records in the above embodiments and will not be repeated here.

[0090] The device includes: a mobile phone, digital camera, tablet computer, or other device with a camera function, or a device with an image processing function, or a device with an image display function. The device may include components such as a memory, a processor, an input unit, a display unit, and a power supply.

[0091] Among them, the memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory may mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as an image playback function, etc.), etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory may also include a memory controller to provide the processor and the input unit with access to the memory.

[0092] The input unit can be used to receive input digital, character, or image information, and generate keyboard, mouse, joystick, optical, or trackball signal input related to user settings and function control. Specifically, the input unit of this embodiment includes not only a camera, but also a touch-sensitive surface (such as a touch display) and other input devices.

[0093] The display unit can be used to display information input by the user or information provided to the user and various graphical user interfaces of the device, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit may include a display panel. Optionally, the display panel can be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), etc. Furthermore, the touch-sensitive surface can cover the display panel. When the touch-sensitive surface detects a touch operation on or near it, it is transmitted to the processor to determine the type of touch event. The processor then provides a corresponding visual output on the display panel based on the type of touch event.

[0094] The embodiment of the present invention further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the memory in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement Figure 1 The computer readable storage medium may be a read-only memory, a disk, or an optical disk.

[0095] The embodiment of the present invention further provides a computer program product, including a computer program / instruction, which is loaded and executed by a processor to implement Figure 1 A method for controlling intelligent environmental equipment based on a wearable brain-computer interface device is shown.

[0096] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. For similar or identical parts between the various embodiments, reference can be made to each other. For the apparatus embodiments, device embodiments, and storage medium embodiments, since they are generally similar to the method embodiments, their descriptions are relatively simple. For relevant parts, reference can be made to the descriptions of the method embodiments.

[0097] Furthermore, in this document, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0098] While the foregoing description shows and describes preferred embodiments of the present invention, it should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments, and can be modified within the scope of the present invention by the teachings herein or by techniques or knowledge in the relevant art. Modifications and variations made by those skilled in the art without departing from the spirit and scope of the present invention are intended to be within the scope of the appended claims.

Claims

1. A method for controlling intelligent environmental equipment based on a wearable brain-computer interface device, characterized in that: The method comprises: The wearable brain-computer interface device collects physiological indicator data of a specific area of the user's body surface, and transmits the physiological indicator data to a data processing terminal in Best Effort mode. The physiological indicator data includes EEG data, photoelectric blood flow mapping data, body temperature data, and motion data. Processing the physiological indicator data according to a preset decision function by the data processing terminal to obtain decision data; Constructing a data flow matrix corresponding to different transmission frequencies for the decision data, and transmitting the data packets encapsulated by the data flow matrix to the corresponding target device through OSC, LSL or a custom upper layer application protocol, wherein the target device includes an intelligent environment device; The intelligent environment device performs corresponding decision operations according to the received data packet.

2. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 1 is characterized in that: The data processing terminal processes the physiological indicator data according to a preset decision function to obtain decision data, including: The data processing terminal performs one or more of time window screening, filtering, time-frequency domain transformation, principal component analysis, and feature extraction on the physiological indicator data to obtain application matrix data; The physiological indicator data or the application matrix data is processed by the decision function at a preset frequency, and the obtained high-order data is used as the decision data; wherein the processing of the decision function includes denoising, filtering and linear / nonlinear mapping in the time domain, frequency domain, spatial domain and code domain.

3. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 1 is characterized in that: The method of transmitting the data packet encapsulated by the data flow matrix to the corresponding target device through OSC, LSL or a custom upper layer application protocol includes: When it is determined that the amplitude or frequency corresponding to the data stream matrix does not match the target device, frequency processing or amplitude processing is performed on the corresponding data stream in the data stream matrix through the data processing terminal or the cloud server; wherein, frequency processing includes performing an up-conversion operation or a down-conversion operation on the frequency of the corresponding data stream in the data stream matrix; amplitude processing includes performing a recalibration operation on the amplitude of the corresponding data stream in the data stream matrix to achieve a target curve within a preset range.

4. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 1 or 3, characterized in that: The target device includes a cloud server; after the data packet encapsulated by the data flow matrix is transmitted to the corresponding target device through OSC, LSL or a custom upper layer application protocol, the method includes: The cloud server performs operations including authentication, storage, processing and distribution according to the received data packet, and sends the data packet after the distribution operation to the data processing terminal or the indirect controlled device in the intelligent environment device.

5. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 1 is characterized in that: The method further comprises: When the local area network formed by the data processing terminal and the smart environment device requires a data stream synchronization service, the smart environment device with timestamp synchronization capability in the local area network is selected as a clock source reference, or the smart environment device with the most accurate clock characteristics is selected as a clock source reference through M2M communication between the smart environment devices; The data processing terminal performs correction processing on the real-time quantity of the clock relative to the reference based on the clock source reference.

6. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 5 is characterized in that: The correction processing of the real-time quantity of the clock relative to the reference based on the clock source by the data processing terminal includes: Adjust the clock by calculating the calibration time; wherein the calibration time t = (T2-T1) + d / 2, d = T4-T1; Where T1 is the timestamp when the client sends the NTP request, T2 is the timestamp when the server receives the NTP request, T3 is the timestamp when the server replies to the NTP request, T4 is the timestamp when the client receives the NTP reply, d1 is the transmission delay of the NTP request packet, and d2 is the transmission delay of the NTP reply packet.

7. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 1 is characterized in that: After the intelligent environment device performs a corresponding decision operation according to the received data packet, the method further includes: The data processing terminal generates a data visualization chart based on the heartbeat data packet sent by the smart environment device, and the data visualization chart is used to display the fluctuation of real-time physiological indicator data.

8. The intelligent environment device control method based on a wearable brain-computer interface device according to claim 1, characterized in that: The wearable brain-computer interface device includes a brain-computer interface device; the data processing terminal includes a computer, mobile phone, tablet, or gateway device with digital signal processing capabilities; the intelligent environment device includes an actuator with sound, light, electricity, magnetism, chemistry, or mechanical motion response capabilities, or a terminal with data processing and display capabilities.

9. An intelligent environmental equipment control device based on a wearable brain-computer interface device, characterized in that: The device comprises: A data acquisition unit, configured to collect physiological indicator data from a specific area of the user's body surface through a wearable brain-computer interface device, and transmit the physiological indicator data to a data processing terminal in Best Effort mode. The physiological indicator data includes EEG data, photoelectric blood flow mapping data, body temperature data, and motion data; a data processing unit, configured to process the physiological indicator data according to a preset decision function through the data processing terminal to obtain decision data; A data stream construction unit is configured to construct a data stream matrix corresponding to different transmission frequencies for the decision data, and transmit the data packets encapsulated by the data stream matrix to the corresponding target device via OSC, LSL or a custom upper layer application protocol, wherein the target device includes an intelligent environment device; An execution unit is configured to perform a corresponding decision operation according to the received data packet through the intelligent environment device.

10. An intelligent environmental device control device based on a wearable brain-computer interface device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory, wherein the computer program is executed by the processor to implement the steps of a smart environment device control method based on a wearable brain-computer interface device as described in any one of claims 1 to 8.

Images