Brain-computer interface high-reliability signal transmission method and system based on hardware-software cooperation

Abstract

The application discloses a brain-computer interface high-reliability signal transmission method and system based on software and hardware cooperation, and relates to the technical field of brain-computer interaction. In the process of pushing the bottom-layer electroencephalogram signal stream, the original hardware absolute serial number and time stamp are injected, and are packaged as a time sequence data stream; the data stream is subjected to feature extraction and classification analysis to obtain electroencephalogram control features; the serial number interval and time stamp corresponding to the trigger features are extracted, and are combined with the control instructions to form a time sequence anchor instruction package; the historical state snapshot in the ring window queue is combined for redundant coding, a UDP cooperative transmission message is generated and sent; after being received by an external execution device, the missing instructions are rearranged based on the serial number and snapshot, and delay interpolation smoothing compensation is carried out based on the time stamp to drive the device. The application overcomes the problem of instruction "ghost swallowing" caused by the superposition of calculation jitter and UDP packet loss under an asynchronous architecture, and realizes zero-error reliable transmission of brain-computer signals under the premise of ensuring extremely low delay.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, specifically to a highly reliable signal transmission method and system for brain-computer interfaces based on hardware and software collaboration. Background Technology

[0002] Brain-computer interface (BCI), as a cutting-edge human-computer interaction technology, can bypass peripheral nerves and body tissues to directly establish a communication and control link between the human brain and external electromechanical devices. Electroencephalography (EEG) is the mainstream non-invasive brainwave acquisition technology. With the iteration of hardware acquisition equipment and the optimization of artificial intelligence decoding algorithms, EEG-based motion intention recognition technology has completed multi-scenario verification in the laboratory and has the potential for industrial transformation in fields such as disability rehabilitation, smart homes, androids, and drone flight assistance guidance.

[0003] In conventional brain-computer interface systems, an "asynchronous pipeline" architecture is typically used: the hardware acquisition device continuously pushes raw signals at a fixed high-frequency sampling rate; the EEG signal extraction software receives the signals, performs feature extraction and algorithm analysis, and generates discrete control commands; finally, in order to meet the requirements of downstream external execution devices for low interaction latency, the system generally uses the UDP protocol to send control commands to the external execution devices.

[0004] However, the aforementioned conventional brain-computer interface systems based on asynchronous architecture have significant technical defects in the underlying signal transmission stage: First, under the asynchronous processing architecture, the computation time of the EEG signal extraction software during complex feature analysis is severely affected by the CPU / GPU resource scheduling of the current operating system. Especially when running high-computational-power-consuming external simulation programs on the same machine, the software's "packaging and sending instructions" action will generate severe underlying computation jitter on the timeline, resulting in a serious disconnect between the hardware and software clocks and extremely uneven transmission rhythm; Second, under high load conditions such as underlying computation jitter, system bus congestion, or inter-process preemption of computing power, using the UDP protocol for transmission is prone to problems such as underlying packet loss or out-of-order arrival of data packets.

[0005] When the two factors mentioned above combine, the user's brain clearly sends an intention signal, and the software successfully parses and packages this "high-value control instruction." However, due to underlying timing misalignment and UDP single-frame packet loss, the instruction is either lost in the transmission channel or discarded by the execution end's state machine due to out-of-order arrival. This phenomenon is called "ghost swallowing" of the instruction. "Ghost swallowing" makes the execution device unable to respond to the user's true intention, undermining the reliability of brain-computer interface control and causing a severe disconnect between the execution operation and the user's brainwave commands, failing to meet the real-time requirements of brain-computer interaction.

[0006] Therefore, this invention proposes a highly reliable signal transmission method and system for brain-computer interfaces based on hardware and software collaboration. Summary of the Invention

[0007] The purpose of this invention is to provide a highly reliable signal transmission method and system for brain-computer interfaces based on hardware and software collaboration. It can overcome the negative impacts of system computational jitter and UDP packet loss and out-of-order delivery under asynchronous architecture without sacrificing the extremely low latency characteristics of the UDP protocol, solve the problem of "ghost swallowing" of brain-computer control commands, and achieve highly reliable signal transmission.

[0008] According to a first aspect of the present invention, in order to achieve the above-mentioned objective, the present invention provides the following technical solution: a highly reliable signal transmission method for a brain-computer interface based on hardware and software collaboration, comprising the following steps: S1. Acquire the raw, continuous EEG signals collected in real time. During the underlying data push process, inject a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw, continuous EEG signal data packet, and encapsulate them to obtain a time-series EEG data stream. S2. Extract the time-series EEG data stream, perform filtering and denoising operations on the time-series EEG data stream based on a preset frequency band range to obtain a denoised time-series EEG signal, perform time-frequency domain transformation on the denoised time-series EEG signal to extract frequency band energy data, construct a multi-dimensional EEG feature vector, input the multi-dimensional EEG feature vector into a pre-constructed classification and analysis model, and perform feature mapping sequentially through the spatial feature aggregation layer, temporal dependency extraction layer and intention mapping fully connected layer in the classification and analysis model to obtain the current EEG control features; S3. When the current EEG control feature meets the preset trigger threshold, extract the hardware absolute sequence number range and the starting hardware timestamp from the time-series EEG data stream corresponding to the triggering current EEG control feature, and merge and encapsulate the hardware absolute sequence number range and the starting hardware timestamp as metadata with the control instructions generated based on the current EEG control feature to obtain a time-series anchored control instruction package. S4. Input the timing anchoring control instruction packet into a pre-built circular sliding window queue, extract the status snapshot containing the sequence number and check code of the historical preset number of sent instructions cached in the circular sliding window queue, save the historical control instruction code, sequence number and CRC, combine the timing anchoring control instruction packet and the status snapshot for joint assembly and redundant encoding, generate a UDP cooperative transmission message, and send the UDP cooperative transmission message to the external execution device through the UDP channel; S5. Obtain the UDP cooperative transmission message received by the external execution device, parse the hardware absolute sequence number carried in the UDP cooperative transmission message, and compare the hardware absolute sequence number with the maximum executed sequence number maintained by the local state machine. If the comparison result is that the serial numbers are not continuous, the historical instructions corresponding to the missing serial numbers are extracted based on the state snapshot parsing, the sequence is rearranged, and the time difference is calculated in combination with the starting hardware timestamp to perform instruction delay interpolation smoothing to obtain the target control instruction. The target control instruction is then sent to the external execution device to drive its execution action.

[0009] Furthermore, the raw, continuous EEG signals acquired in real time are injected with a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw, continuous EEG signal during the underlying data streaming process, and encapsulated to obtain a time-series EEG data stream, as follows: S1.1 Acquire the analog EEG voltage signal collected by the front-end electrode sensor, and then convert it to a preset sampling frequency via an analog-to-digital converter. The simulated EEG voltage signal is discretized, sampled, and quantized to obtain a digital EEG data matrix containing multiple sampling points. ,in Index of discrete-time sampling points; S1.2 Process the digital EEG data matrix according to the preset data frame length. Low-level memory segmentation is performed to obtain continuous single-frame EEG data. In each frame of the single-frame EEG data When writing to the transmit buffer, an incrementing counter in the underlying hardware register is triggered to generate a hardware absolute sequence number corresponding to the current frame. The absolute serial number of the hardware satisfies a discrete recursive equation: in, This is the frame sequence number of the current data packet, and the base sequence number is defined during system power-on initialization. ; S1.3 Within the same hardware clock cycle that triggers the incrementing counter, the current cumulative oscillation cycle count value of the local hardware crystal oscillator is synchronously read. Combined with the inherent nominal frequency of the hardware crystal oscillator, the current frame sequence number is calculated. Strictly physically aligned hardware timestamps The calculation formula is as follows: in, This is the reference zero-point time recorded during the system synchronization handshake. For the first The accumulated crystal oscillator count value read at the frame trigger moment. The inherent nominal frequency of the hardware crystal oscillator; S1.4 According to the preset communication bus protocol bit width, the absolute serial number of the hardware is... With the hardware timestamp The kernel protocol header segment allocated to the underlying data packet, and the single frame of EEG data... The data payload segment is mapped to the underlying data packet, and a cyclic redundancy check code is appended to the end of the packet. The packets are then concatenated bit by bit to form a complete time-series data packet. Multiple time-series data packets generated consecutively are then transmitted and pushed upwards in sequence to output the time-series EEG data stream.

[0010] Furthermore, the current brainwave control features are obtained based on temporal EEG data stream processing, and the specific steps are as follows: S2.1 Parse the data payload segment of the time-series EEG data stream and extract the data containing... The original multichannel EEG sequence for each lead channel, targeting any lead channel The EEG sequence of this channel is filtered and denoised using a preset infinite impulse response bandpass filter to obtain the denoised temporal EEG signal. The corresponding time-domain linear constant-coefficient difference equation is: in, For discrete-time sampling point index, For lead channel numbering, For the first segment extracted from the payload segment The current input sample value for the channel. and These are the orders of the forward and feedback filters, respectively. For the first Forward feed coefficient, For the first The passband cutoff range of the bandpass filter is set according to the preset EEG characteristic frequency band range parameter, which is the first-order feedback coefficient. S2.2 Set the sliding time window length to And the window movement step length is The denoised temporal EEG signals of each lead channel Simultaneously perform short-time Fourier transform and calculate the target frequency band. Cumulative frequency band energy data within The calculation formula is as follows: in, This is the index of the current window frame of the sliding time window. For discrete frequency indexing, Hamming weighted window function to suppress spectral leakage, and The indexes of the lower and upper frequency limits of the preset frequency band range in the discrete spectrum are respectively used to index the current window frame. All The accumulated frequency band energy data of each channel are spliced ​​together according to the lead space topology to construct the multidimensional EEG feature vector. ; S2.3 The multidimensional EEG feature vector Input the spatial feature aggregation layer of the classification parsing model, and pass it through a pre-trained spatial convolution weight matrix. Nonlinear feature aggregation of spatial dimensions between channels is performed to reduce dimensionality and obtain spatial fusion features. : The spatial fusion features are then... The temporal dependency extraction layer of the classification parsing model is sequentially input, combined with the hidden state of the previous window frame. Recursively compute the execution temporal context to obtain fused intent features that include dependencies on historical action sequences. : in, , , These are the spatial convolution weight matrix, the temporal input weight matrix, and the temporal recursive weight matrix, respectively. and For the corresponding bias vector, and All are non-linear activation functions; S2.4 The fusion intent features The input intent mapping fully connected layer is used to calculate the state probability distribution, and the probability vector of the current state belonging to each preset intent category is output through a normalized exponential function. and by taking the maximum probability index function from the probability vector Extract the corresponding classification labels as the final current EEG control features. : in, This is the mapping matrix for the fully connected layer. This is the bias vector for the fully connected layer. This represents the total number of preset categories of external device control actions. The probability vector The first in A probability element, the current brainwave control feature Used to characterize the current discrete action intent of the target user.

[0011] Furthermore, the pre-built classification parsing model includes the following steps in its construction and training process: Acquire historical continuous EEG data carrying the true intention action tags of external devices, and perform filtering, noise reduction and time-frequency domain feature extraction operations on the historical continuous EEG data to obtain a historical multidimensional EEG feature vector sequence. The true intention action labels are converted into true probability distribution vectors in one-hot encoded format. The historical multidimensional EEG feature vector sequence is then aligned with the corresponding true probability distribution vectors to construct a training sample set. in For sample time step index, For the first A historical multidimensional EEG feature vector at each time step, This represents the corresponding true probability distribution vector. The total number of training samples; An initial classification and parsing model is constructed, using a normal distribution to randomly initialize the spatial convolutional weight matrix and bias vector of the spatial feature aggregation layer, the temporal input weight matrix, the temporal recursive weight matrix and bias vector of the temporal dependency extraction layer, and the fully connected layer mapping matrix and bias vector of the intent mapping fully connected layer. This is achieved by using the historical multidimensional EEG feature vector sequence from the training sample set. The initial classification model is input according to a preset batch size, and then sequentially passes through the dimensionality reduction operation of the spatial feature aggregation layer, the historical state recursive operation of the temporal dependency extraction layer, and the normalized exponential mapping of the intent mapping fully connected layer, outputting the index of the sample time step. The corresponding predicted probability distribution vector ; Extract the predicted probability distribution vector and the true probability distribution vector corresponding to the training sample set. The cross-entropy cost function is used to calculate the global classification loss value of the forward propagation of the current batch. The calculation formula is as follows: in, This refers to the number of samples in a single training iteration. For the total number of intended categories, For the first batch The th sample in the true probability distribution vector One element, For the first batch The th sample in the predicted probability distribution vector One element; Based on the calculated global classification loss value The global classification loss value is calculated using a backpropagation algorithm based on the chain rule. The loss gradient of each weight matrix and bias vector in the initial classification parsing model; An adaptive moment estimation optimization algorithm is used to iteratively update the weight matrix and bias vector for each term based on the loss gradient. The weight parameter update formula is as follows: in, Represents any weight matrix or bias vector to be updated. For the current iteration round, The preset base learning rate, This is the first-order momentum estimate after bias correction for the current iteration. This is the second-order momentum estimate after bias correction for the current iteration round. To prevent the smoothing constant from having a denominator of zero; Repeat the above training steps until the global classification loss value is reached. If the value is less than the preset convergence threshold or the maximum number of iterations is reached, stop updating the parameters and solidify all the weight matrices and bias vectors, and output the pre-built classification parsing model that has been trained and converged.

[0012] Further, the absolute hardware sequence number range and the starting hardware timestamp are extracted from the temporal EEG data stream corresponding to the triggering current EEG control feature. The absolute hardware sequence number range and the starting hardware timestamp are then merged and encapsulated as metadata with the control instructions generated based on the current EEG control feature to obtain a temporal anchoring control instruction package, specifically including: S3.1 Based on the probability vector Extract the maximum classification probability value. The maximum classification probability value Compared with the preset intent confidence threshold Perform numerical comparison, and when a judgment is made At that time, confirm the current brainwave control characteristics. If the preset trigger threshold condition is met, and the preset device action command mapping dictionary is extracted, the current EEG control features are mapped according to the unique mapping relationship. Transformed into specific control command encoding for the external device ; S3.2 Obtain the features used to extract the current EEG control features Current sliding time window index Combined with the preset sliding time window length Window moving step length and the number of fixed sampling points contained in a single frame data packet in the time-series EEG data stream. Calculate the hardware absolute sequence number range that triggers the current EEG control feature. The calculation formula is as follows: in, The reference sequence number is set during system power-on initialization. This is the floor function. The starting sequence number of the interval. This is the end sequence number of the interval; Synchronously query the time-series EEG data stream and extract the sequence number corresponding to the starting sequence number of the interval. The hardware timestamp recorded in the kernel protocol header segment of the corresponding data packet is used as the starting hardware timestamp. ; S3.3 Calculate the hardware absolute serial number range and the starting hardware timestamp The data is divided into time-series metadata fields, and these time-series metadata fields are concatenated to the specific control instruction encoding. The head position is determined, and a check bit calculated based on a preset generator polynomial is added to the tail. These are then combined and encapsulated to form the timing anchoring control instruction package. The data structure expression of the timing anchoring control instruction packet is as follows: in, This represents a concatenation of consecutive bit streams of data fields in memory. This is a cyclic redundancy check sequence calculated for the entire current instruction packet.

[0013] Further, the timing anchoring control instruction packet is input into a pre-built circular sliding window queue. A state snapshot containing the sequence number and checksum of a preset number of historically transmitted instructions is extracted from the circular sliding window queue. The timing anchoring control instruction packet and the state snapshot are then jointly assembled and redundantly encoded to generate a UDP cooperative transmission message. This UDP cooperative transmission message is then sent to an external execution device via a UDP channel, as detailed below: S4.1 The timing anchoring control instruction package Push into a pre-built circular sliding window queue The position of the tail pointer when the circular sliding window queue The current storage length exceeds the preset maximum window capacity. At that time, the earliest historical command packet at the head of the queue will pop up; From the circular sliding window queue Extract the preceding order of the current tail pointer. The state snapshot is constructed by extracting the interval start sequence number and cyclic redundancy check sequence corresponding to each historical frame from the historical frames. Its mathematical expression is: in, This represents a structure array with a fixed length. This represents a concatenation of consecutive bit streams of data fields in memory. The preset number of historical frames and the preset size are satisfied. , and The first The starting sequence number and check code of the interval corresponding to the frame history instruction; S4.2 The timing anchoring control instruction package With the aforementioned state snapshot Perform bitwise concatenation to obtain a combined data block. Extract the preset forward error correction coding matrix Through the forward error correction coding matrix For the joint data block Perform Galois field multiplication redundancy coding operations to generate corresponding redundancy check blocks. and the joint data block With the redundant check block Merging and assembling to generate transmission payload The calculation formula is as follows: in, Indicates the presupposed Galois domain Finite field matrix multiplication, To generate the linear generator matrix of the system error-correcting code, A redundant check byte stream used to combat packet loss during network transmission; S4.3 According to the underlying standard of the connectionless User Datagram Protocol, the transmitted payload is... The UDP cooperative transmission message is encapsulated by appending a transport layer header consisting of the source port and the destination port, a network layer header containing the IP address of the destination external execution device, and a data link layer physical frame header in sequence. Its data structure expression is: in, , , These are the standard communication header segments for the Media Access Control layer, Internet Protocol layer, and User Datagram Protocol layer, respectively; finally, by calling the underlying network interface card hardware driver, the continuously generated UDP cooperative transmission messages are... The data is sent asynchronously to the external execution device via a UDP channel.

[0014] Furthermore, the external execution devices include embodied robots, aircraft, detectors, immersive gaming / XR devices, and virtual engine environments.

[0015] Furthermore, in step S5, the generation of the target control command is specifically as follows: S5.1 Obtain the UDP cooperative transmission message received by the external execution device at the transport layer, perform reverse data unpacking on the UDP cooperative transmission message, and extract the current hardware absolute sequence number range carried in the payload of the current message. Current starting hardware timestamp and current control commands Simultaneously, it reads the maximum executed sequence number maintained in the local receiving state machine memory of the external execution device. The starting serial number in the current hardware absolute serial number range With the maximum sequence number already executed Perform numerical calculations to determine the difference in serial numbers. ; S5.2 If the difference in the serial numbers is determined If the comparison result indicates that the sequence number is discontinuous, the redundant appended state snapshot in the UDP cooperative transmission message is extracted. The state snapshot contains historical data. Frame control commands and corresponding sequence number mapping table ; Based on the set of missing serial numbers In the mapping table A matching search is performed to extract the missing historical instructions, and the extracted historical instructions are compared with the current control instructions. The memory addresses are rearranged according to the monotonically increasing absolute sequence number to construct a sequentially continuous queue of instructions to be executed. ; S5.3 For the queue of instructions to be executed The instruction in the command retrieves the current system rendering clock of the external execution device. Combined with the current starting hardware timestamp Calculate the time difference of end-to-end transmission scheduling of instructions. An exponential decay interpolation algorithm is used based on the time difference. The formula for smoothing compensation of command control parameters is as follows: in, The target control command is after smooth compensation. For the queue of instructions to be executed The original command parameters for the current dequeue. These are the historical execution instruction parameters cached by the external execution device in the previous machine response cycle. The adaptive interpolation weights are dynamically calculated based on the time difference. The preset external execution device smoothing attenuation constant; S5.4 The target control command obtained after the smoothing compensation process. The instructions are sequentially sent to the underlying driver controller of the external execution device to drive the external execution device to execute the corresponding continuous actions. After all actions in the current instruction queue have been executed, the maximum executed sequence number maintained by the local receiving state machine is transferred. Synchronously updated to the termination serial number within the current hardware absolute serial number range. This completes the state iteration closed loop of the single-interaction control state machine.

[0016] According to a second aspect of the present invention, the present invention provides a high-reliability signal transmission system for a brain-computer interface based on hardware and software collaboration, for implementing a high-reliability signal transmission method for a brain-computer interface based on hardware and software collaboration as described in the first aspect, comprising: The data acquisition and timing encapsulation module is used to acquire raw continuous EEG signals in real time. During the underlying data push process, it injects a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw continuous EEG signal data packet, and encapsulates it to obtain a timing EEG data stream. The signal preprocessing and feature parsing module is used to extract the time-series EEG data stream, perform filtering and denoising operations on the time-series EEG data stream based on a preset frequency band range to obtain a denoised time-series EEG signal, perform time-frequency domain transformation on the denoised time-series EEG signal to extract frequency band energy data, construct a multi-dimensional EEG feature vector, input the multi-dimensional EEG feature vector into a pre-constructed classification parsing model, and perform feature mapping sequentially through the spatial feature aggregation layer, temporal dependency extraction layer and intention mapping fully connected layer in the classification parsing model to obtain the current EEG control features; The timing anchoring instruction encapsulation module is used to extract the hardware absolute sequence number range and the starting hardware timestamp from the timing EEG data stream corresponding to the current EEG control feature when the current EEG control feature meets the preset trigger threshold, and to merge and encapsulate the hardware absolute sequence number range and the starting hardware timestamp as metadata with the control instruction generated based on the current EEG control feature to obtain a timing anchoring control instruction package. The redundant snapshot cooperative transmission module is used to input the timing anchoring control instruction packet into a pre-built circular sliding window queue, extract the status snapshots containing the sequence numbers and check codes of the historical preset number of sent instructions cached in the circular sliding window queue, save the historical control instruction codes, sequence numbers and CRCs, jointly assemble the timing anchoring control instruction packet and the status snapshots and perform redundant encoding to generate a UDP cooperative transmission message, and send the UDP cooperative transmission message to an external execution device through a UDP channel; The receiving rearrangement and smooth execution module is used to obtain the UDP cooperative transmission message received by the external execution device, parse the hardware absolute sequence number carried in the UDP cooperative transmission message, and compare the hardware absolute sequence number with the maximum executed sequence number maintained by the local state machine. If the comparison result is that the serial numbers are not continuous, the historical instructions corresponding to the missing serial numbers are extracted based on the state snapshot parsing, the sequence is rearranged, and the time difference is calculated in combination with the starting hardware timestamp to perform instruction delay interpolation smoothing to obtain the target control instruction. The target control instruction is then sent to the external execution device to drive its execution action.

[0017] According to a third aspect of the present invention, a terminal device is provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor, and when the processor loads and executes the computer program, it employs the high-reliability signal transmission method for brain-computer interface based on hardware and software collaboration described in the first aspect.

[0018] This invention has at least the following beneficial effects: 1. This invention injects a monotonically increasing absolute sequence number and a high-precision timestamp natively into the underlying hardware streaming end, and accurately extracts this timing certificate at the software parsing end. It then encapsulates this as metadata and control instructions. This deep hardware and software collaboration mechanism enables the highest-level brain control instructions to be seamlessly and accurately traced back to the lowest-level physical trigger moment, effectively reducing the jitter in feature parsing calculations caused by the preemption of operating system background resources, and ensuring the consistency of timing throughout the entire chain.

[0019] 2. This invention introduces a redundant cooperative coding transmission mechanism based on a circular sliding window and historical state snapshots. This mechanism eliminates the latency spikes caused by the traditional TCP protocol timeout retransmission mechanism, enabling the execution end to directly rely on the compact snapshots in subsequent packets to restore and rearrange instructions when facing network packet loss. While maintaining the extremely low latency characteristics of UDP, it achieves highly reliable transmission and delivery of high-value intent instructions.

[0020] 3. This invention constructs a state machine model with deeply coupled hardware timestamps at the receiving end of the external execution device. When the system faces extreme network disorder or sudden changes in instruction latency, the execution end no longer mechanically executes delayed instructions. Instead, it dynamically calculates the exponential decay weight based on the physical time difference between the source starting hardware timestamp and the current system rendering clock, and performs adaptive interpolation smoothing compensation on the instruction control parameters. This effectively filters out the stuttering of external physical device actions and the dizziness caused by the non-uniform arrival of instructions, as well as the dizziness of virtual engine screen rendering, thus creating a smooth, natural, and highly immersive closed-loop brain-computer interaction experience for the target user.

[0021] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the overall process of the method described in this invention. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Example 1: Please see Figure 1 This invention provides a technical solution: a highly reliable signal transmission method for brain-computer interfaces based on hardware and software collaboration, comprising the following steps: S1. Acquire raw, continuous EEG signals acquired in real time. During the underlying data streaming process, inject a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw, continuous EEG signal data packet, and encapsulate it to obtain a time-series EEG data stream, specifically including: S1.1 Acquire the analog EEG voltage signal collected by the front-end electrode sensor, and then convert it to a preset sampling frequency via an analog-to-digital converter. The simulated EEG voltage signal is discretized, sampled, and quantized to obtain a digital EEG data matrix containing multiple sampling points. ,in Index of discrete-time sampling points; S1.2 The digital EEG data matrix is ​​processed according to the preset data frame length. Low-level memory segmentation is performed to obtain continuous single-frame EEG data. In each frame of single-frame EEG data When writing to the transmit buffer, an incrementing counter in the underlying hardware register is triggered to generate a hardware absolute sequence number corresponding to the current frame. The absolute serial number of the hardware satisfies the discrete recursive equation: in, This is the frame sequence number of the current data packet, and the base sequence number is defined during system power-on initialization. ; S1.3 Within the same hardware clock cycle that triggers the incrementing counter, synchronously read the current cumulative oscillation cycle count value of the local hardware crystal oscillator, and calculate the value corresponding to the current frame number by combining it with the inherent nominal frequency of the hardware crystal oscillator. Strictly physically aligned hardware timestamps The calculation formula is as follows: in, This is the reference zero-point time recorded during the system synchronization handshake. For the first The accumulated crystal oscillator count value read at the frame trigger moment. This is the inherent nominal frequency of the hardware crystal oscillator; S1.4 According to the preset communication bus protocol bit width, the hardware absolute serial number is... Hardware timestamp The kernel protocol header segment allocated to the underlying data packet, and the single frame of EEG data The data payload segment is mapped to the underlying data packet, and a cyclic redundancy check code is appended to the end of the packet. The packets are then concatenated bit by bit to form a complete time-series data packet. Multiple consecutively generated time-series data packets are then transmitted and pushed upwards in sequence to output a time-series EEG data stream. S2. Extract the time-series EEG data stream, perform filtering and denoising operations on the time-series EEG data stream based on a preset frequency band range to obtain a denoised time-series EEG signal, perform time-frequency domain transformation on the denoised time-series EEG signal to extract frequency band energy data, construct a multi-dimensional EEG feature vector, input the multi-dimensional EEG feature vector into a pre-constructed classification and parsing model, and perform feature mapping sequentially through the spatial feature aggregation layer, temporal dependency extraction layer, and intention mapping fully connected layer in the classification and parsing model to obtain the current EEG control features, specifically including: S2.1 Parse the data payload segment of the time-series EEG data stream and extract the data containing... The original multichannel EEG sequence for each lead channel, targeting any lead channel The EEG sequence of this channel was filtered and denoised using a preset infinite impulse response bandpass filter to obtain a denoised temporal EEG signal. The corresponding time-domain linear constant-coefficient difference equation is: in, For discrete-time sampling point index, For lead channel numbering, To extract the first from the effective payload segment The current input sample value for the channel. and These are the orders of the forward and feedback filters, respectively. For the first Forward feed coefficient, For the first The first-order feedback coefficient and the passband cutoff range of the bandpass filter are set according to the preset EEG characteristic frequency band range parameters; S2.2 Set the sliding time window length to And the window movement step length is Denoising of temporal EEG signals from each lead channel Simultaneously perform short-time Fourier transform and calculate the target frequency band. Cumulative frequency band energy data within The calculation formula is as follows: in, This is the index of the current window frame of the sliding time window. For discrete frequency indexing, Hamming weighted window function to suppress spectral leakage, and The indexes of the lower and upper frequency limits of the preset frequency band range in the discrete spectrum are respectively used to index the current window frame. All The accumulated frequency band energy data of each channel are spliced ​​together according to the lead space topology to construct a multidimensional EEG feature vector. ; S2.3 Multidimensional EEG Feature Vector The spatial feature aggregation layer of the input classification model is processed by a pre-trained spatial convolution weight matrix. Nonlinear feature aggregation of spatial dimensions between channels is performed to reduce dimensionality and obtain spatial fusion features. : Then, spatial fusion features The temporal dependency extraction layer of the sequential input classification parsing model is combined with the hidden state of the previous window frame. Recursively compute the execution temporal context to obtain fused intent features that include dependencies on historical action sequences. : in, , , These are the spatial convolution weight matrix, the temporal input weight matrix, and the temporal recursive weight matrix, respectively. and For the corresponding bias vector, and All are non-linear activation functions; S2.4 will fuse intent features The input intent mapping fully connected layer calculates the state probability distribution and outputs a probability vector of the current state belonging to each preset intent category through a normalized exponential function. And by taking the maximum probability index function from the probability vector Extract the corresponding classification labels as the final current EEG control features. : in, This is the mapping matrix for the fully connected layer. This is the bias vector for the fully connected layer. This represents the total number of preset categories of external device control actions. Probability vector The first in Each probability element represents the current brainwave control characteristics. Used to characterize the current discrete action intent of the target user; It should be further explained that the pre-built classification parsing model includes the following steps in its construction and training process: (1) Obtain historical continuous EEG data carrying the true intention action label of external device, perform filtering and noise reduction and time-frequency domain feature extraction on the historical continuous EEG data to obtain historical multidimensional EEG feature vector sequence; The true intention action labels are converted into true probability distribution vectors in one-hot encoding format. Historical multidimensional EEG feature vector sequences are then aligned with their corresponding true probability distribution vectors to construct a training sample set. in For sample time step index, For the first A historical multidimensional EEG feature vector at each time step, This represents the corresponding true probability distribution vector. The total number of training samples; (2) Construct an initial classification and parsing model. Use a normal distribution to randomly initialize the spatial convolution weight matrix and bias vector of the spatial feature aggregation layer, the temporal input weight matrix of the temporal dependency extraction layer, the temporal recursive weight matrix and bias vector, and the fully connected layer mapping matrix and bias vector of the intention mapping fully connected layer. The historical multidimensional EEG feature vector sequence in the training sample set is used. The initial classification model is input according to the preset batch size. It then undergoes dimensionality reduction in the spatial feature aggregation layer, recursive historical state operations in the temporal dependency extraction layer, and normalized exponential mapping in the intent mapping fully connected layer. The output is the index of the sample time step. The corresponding predicted probability distribution vector ; (3) Extract the predicted probability distribution vector and the true probability distribution vector corresponding to the training sample set. The cross-entropy cost function is used to calculate the global classification loss value of the forward propagation of the current batch. The calculation formula is as follows: in, This refers to the number of samples in a single training iteration. For the total number of intended categories, For the first batch The th sample in the true probability distribution vector One element, For the first batch The th sample in the predicted probability distribution vector One element; (4) Based on the calculated global classification loss value The backpropagation algorithm based on the chain rule is applied to calculate the global classification loss value. The loss gradient for each weight matrix and bias vector in the initial classification parsing model; An adaptive moment estimation optimization algorithm is used, which iteratively updates the weight matrices and bias vectors based on the loss gradient. The formula for updating the weight parameters is as follows: in, Represents any weight matrix or bias vector to be updated. For the current iteration round, The preset base learning rate, This is the first-order momentum estimate after bias correction for the current iteration. This is the second-order momentum estimate after bias correction for the current iteration round. To prevent the smoothing constant from having a denominator of zero; (5) Repeat the above training steps (1) to (4) until the global classification loss value is reached. If the value is less than the preset convergence threshold or the maximum iteration round condition is reached, stop updating the parameters and solidify all weight matrices and bias vectors, and output the pre-built classification parsing model that has been trained and converged. It should be further noted that the spatial feature aggregation layer is preferably constructed using a one-dimensional convolutional network. Local spatial features are extracted along the lead dimension through convolutional kernels. Different convolutional layers are connected by non-linear activation functions, and the output is a dimensionality-reduced spatial fusion feature. The temporal dependency extraction layer is preferably implemented using a recurrent neural network (RNN) structure, but GRU or LSTM can also be used. S3. When the current EEG control feature meets the preset trigger threshold, extract the hardware absolute sequence number range and the starting hardware timestamp from the time-series EEG data stream corresponding to the triggering current EEG control feature. Then, merge and encapsulate the hardware absolute sequence number range and the starting hardware timestamp as metadata with the control instructions generated based on the current EEG control feature to obtain a time-series anchored control instruction package, as follows: S3.1 Based on probability vectors Extract the maximum classification probability value. The maximum classification probability value Compared with the preset intent confidence threshold Perform numerical comparison, and when a judgment is made At that time, confirm the current brainwave control characteristics. If the preset trigger threshold conditions are met, and the preset device action command mapping dictionary is extracted, the current EEG control features are mapped according to the unique mapping relationship. Transformed into specific control command codes for external devices ; Among them, the device action command mapping dictionary is pre-established based on the control protocol supported by the external execution device, and each EEG control feature corresponds to a unique control command code; S3.2 Obtain features for extracting current EEG control. Current sliding time window index Combined with the preset sliding time window length Window moving step length And the number of fixed sampling points contained in a single frame data packet in a time-series EEG data stream. Calculate the hardware absolute sequence number range that triggers the current brainwave control features. The calculation formula is as follows: in, The reference sequence number is set during system power-on initialization. This is the floor function. The starting sequence number of the interval. This is the end sequence number of the interval; Synchronously query the time-series EEG data stream and extract the interval start sequence number. The hardware timestamp recorded in the kernel protocol header segment of the corresponding data packet is used as the starting hardware timestamp. ; S3.3 Calculate the hardware absolute serial number range and the starting hardware timestamp The data is divided into time-series metadata fields, and these fields are then concatenated with the specific control instruction codes. The header position is determined, and a check bit calculated based on a preset generator polynomial is added to the tail. These are then combined and encapsulated to form the timing anchoring control instruction package. The data structure expression for the timing anchoring control instruction packet is as follows: in, This represents a concatenation of consecutive bit streams of data fields in memory. This is the cyclic redundancy check sequence calculated for the entire current instruction packet; S4. Input the timing anchoring control command packet into a pre-built circular sliding window queue, extract the status snapshot containing the sequence number and checksum of the previously sent commands for a preset number of frames from the circular sliding window queue, save the historical control command encoding, sequence number, and cyclic redundancy check (CRC), jointly assemble the timing anchoring control command packet and the status snapshot, perform redundancy encoding, generate a UDP cooperative transmission message, and send the UDP cooperative transmission message to the external execution device through the User Datagram Protocol (UDP) channel. Specifically, this includes: S4.1 Timing anchor control instruction packet Push into a pre-built circular sliding window queue The position of the tail pointer in the circular sliding window queue. The current storage length exceeds the preset maximum window capacity. At that time, the earliest historical command packet at the head of the queue will pop up; From the circular sliding window queue Extract the preceding order of the current tail pointer. From each historical frame, the starting sequence number of the interval and the cyclic redundancy check sequence corresponding to each historical frame are extracted to construct a state snapshot. Its mathematical expression is: in, This represents a structure array with a fixed length. This represents a concatenation of consecutive bit streams of data fields in memory. The preset number of historical frames and the preset size are satisfied. , and The first The starting sequence number and check code of the interval corresponding to the frame history instruction; S4.2 Anchor the timing control command packet With state snapshot Perform bitwise concatenation to obtain a combined data block. Extract the preset forward error correction coding matrix Through the forward error correction coding matrix For combined data blocks Perform Galois field multiplication redundancy coding operations to generate corresponding redundancy check blocks. and combine data blocks With redundant check block Merging and assembling to generate transmission payload The calculation formula is as follows: in, Indicates the presupposed Galois domain Finite field matrix multiplication, To generate the linear generator matrix of the system error-correcting code, A redundant check byte stream used to combat packet loss during network transmission; Among them, the forward error correction coding matrix The Reed-Solomon coding generation matrix is ​​preferably used, but other linear block codes can also be used to generate the matrix. This embodiment does not make specific limitations here, and the choice can be made according to the actual situation. However, no matter what is chosen, it falls within the protection scope of this embodiment. S4.3 In accordance with the underlying standard of the connectionless User Datagram Protocol, for transmitting the payload... The transport layer header consisting of the source port and destination port, the network layer header containing the IP address of the destination external execution device, and the data link layer physical frame header are appended sequentially to the header to encapsulate and generate a UDP cooperative transmission message. Its data structure expression is: in, , , These are the standard communication header segments for the Media Access Control layer, Internet Protocol layer, and User Datagram Protocol layer, respectively; finally, by calling the underlying network interface card hardware driver, the continuously generated UDP cooperative transmission messages are transmitted. It is sent asynchronously to the external execution device via the UDP channel in sequence; The external execution devices include robots, aircraft, detectors, immersive gaming / VR devices, and virtual engine environments, which are not specifically limited in this embodiment; S5. Obtain the UDP cooperative transmission message received by the external execution device, parse the hardware absolute sequence number carried in the UDP cooperative transmission message, and compare the hardware absolute sequence number with the maximum executed sequence number maintained by the local state machine. If the comparison result shows that the sequence numbers are not consecutive, the historical instructions corresponding to the missing sequence numbers are extracted based on the state snapshot parsing, the sequence is rearranged, and the time difference is calculated by combining the starting hardware timestamp to perform instruction delay interpolation smoothing to obtain the target control instruction. The target control instruction is then sent to the external execution device to drive its execution action, specifically including: S5.1 Obtain the UDP cooperative transport message received by the external execution device at the transport layer, perform reverse data unpacking on the UDP cooperative transport message, and extract the current hardware absolute sequence number range carried in the payload of the current message. Current starting hardware timestamp and current control commands Simultaneously, it reads the maximum executed sequence number maintained in the local receiving state machine memory of the external execution device. The starting serial number in the current hardware absolute serial number range With the highest executed sequence number Perform numerical calculations to determine the difference in serial numbers. ; S5.2 If the difference in serial numbers is determined If the comparison result indicates that the sequence number is discontinuous, then the redundant additional state snapshots in the UDP cooperative transmission messages are extracted. The state snapshots contain history. Frame control commands and corresponding sequence number mapping table ; Based on the set of missing serial numbers In the mapping table Perform a matching search to extract missing historical instructions, and then match the extracted historical instructions with the current control instructions. The memory addresses are rearranged according to the monotonically increasing absolute sequence number to construct a sequentially continuous queue of instructions to be executed. ; S5.3 For the queue of instructions to be executed The instruction in the command retrieves the current system rendering clock of the external execution device. Combined with the current starting hardware timestamp Calculate the time difference of end-to-end transmission scheduling of instructions. An exponential decay interpolation algorithm based on time difference is used. The formula for smoothing compensation of command control parameters is as follows: in, The target control command after smooth compensation. Queue of instructions to be executed The original command parameters for the current dequeue. This is for the historical execution instruction parameters cached by the external execution device in the previous machine response cycle. The adaptive interpolation weights are dynamically calculated based on the time difference. The preset external execution device smoothing attenuation constant; Among them, the exponential decay interpolation weight decreases monotonically as the time difference increases, causing historical control parameters to gradually exit the current control result; S5.4 The target control command obtained after smoothing compensation processing The instructions are sequentially sent to the underlying driver controller of the external execution device to drive the external execution device to perform the corresponding continuous actions. After all actions in the current instruction queue have been executed, the maximum sequence number of the executed sequence maintained by the local receiving state machine is transferred. Synchronously update to the end serial number within the current hardware absolute serial number range. This completes the state iteration closed loop of the single-interaction control state machine.

[0026] In summary, this invention injects a monotonically increasing absolute sequence number and a high-precision timestamp natively into the underlying hardware streaming end, and accurately extracts this timing certificate at the software parsing end. This certificate is then used as metadata and jointly encapsulated with control instructions. This deep hardware and software collaboration mechanism enables the highest-level brain control instructions to be smoothly and accurately traced back to the lowest-level physical trigger moment. This effectively reduces the jitter in feature parsing calculations caused by the preemption of operating system background resources (such as high-load rendering engine operation), and ensures the consistency of the timing throughout the entire chain. Furthermore, this invention innovatively introduces a redundant cooperative coding transmission mechanism based on a circular sliding window and historical state snapshots. This mechanism eliminates the latency spikes caused by the traditional TCP protocol timeout retransmission mechanism, enabling the execution end to directly rely on the compact snapshots in subsequent packets to achieve instruction-level recovery and seamless reordering when facing network packet loss. While maintaining the extremely low latency characteristics of UDP, it achieves highly reliable transmission and delivery of high-value intent instructions.

[0027] Example 2: This embodiment provides a highly reliable signal transmission system for a brain-computer interface based on hardware and software collaboration, used to implement the highly reliable signal transmission method for a brain-computer interface based on hardware and software collaboration described in Embodiment 1, including: The data acquisition and timing encapsulation module is used to acquire raw continuous EEG signals in real time. During the underlying data push process, it injects a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw continuous EEG signal data packet, and encapsulates it to obtain a timing EEG data stream. The signal preprocessing and feature parsing module is used to extract the time-series EEG data stream. Based on a preset frequency band range, it performs filtering and denoising operations on the time-series EEG data stream to obtain a denoised time-series EEG signal. It performs time-frequency domain transformation on the denoised time-series EEG signal to extract frequency band energy data, constructs a multi-dimensional EEG feature vector, and inputs the multi-dimensional EEG feature vector into a pre-constructed classification parsing model. The feature is then mapped sequentially through the spatial feature aggregation layer, the temporal dependency extraction layer, and the intention mapping fully connected layer in the classification parsing model to obtain the current EEG control features. The timing anchoring instruction encapsulation module is used to extract the hardware absolute sequence number range and the starting hardware timestamp from the timing EEG data stream corresponding to the current EEG control feature when the current EEG control feature meets the preset trigger threshold. The hardware absolute sequence number range and the starting hardware timestamp are then merged and encapsulated with the control instruction generated based on the current EEG control feature as metadata to obtain the timing anchoring control instruction package. The redundant snapshot cooperative transmission module is used to input the timing anchor control command packet into a pre-built circular sliding window queue, extract the status snapshots containing the sequence number and check code of the historical preset number of sent commands cached in the circular sliding window queue, save the historical control command code, sequence number and CRC, jointly assemble and redundantly encode the timing anchor control command packet and the status snapshot, generate UDP cooperative transmission message, and send the UDP cooperative transmission message to the external execution device through the UDP channel; The receiving rearrangement and smooth execution module is used to obtain UDP cooperative transmission messages received by external execution devices, parse the hardware absolute sequence number carried in the UDP cooperative transmission message, and compare the hardware absolute sequence number with the maximum executed sequence number maintained by the local state machine. If the comparison result shows that the serial numbers are not continuous, the historical instructions corresponding to the missing serial numbers are extracted based on the status snapshot parsing, the sequence is rearranged, and the time difference is calculated by combining the starting hardware timestamp to perform instruction delay interpolation smoothing to obtain the target control instruction. The target control instruction is then sent to the external execution device to drive its execution action.

[0028] Example 3: The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores the computer program capable of running on the processor. When the processor loads and executes the computer program, it employs the high-reliability signal transmission method for brain-computer interface based on hardware and software collaboration described in Embodiment 1.

[0029] It should be noted that the terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server, and the terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may also include input / output devices, network access devices, and buses.

[0030] Furthermore, the processor can be a central processing unit (CPU). Of course, depending on the actual use, other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be used. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.

[0031] In the description of this specification, the references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0032] The foregoing has shown and described the basic principles, main features, and advantages of this application. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of this application. Various changes and modifications can be made to this application without departing from the spirit and scope thereof, and all such changes and modifications fall within the scope of the claims of this application.

Claims

1. A highly reliable signal transmission method for brain-computer interfaces based on hardware and software collaboration, characterized in that, Includes the following steps: S1. Acquire the raw, continuous EEG signals collected in real time. During the underlying data push process, inject a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw, continuous EEG signal data packet, and encapsulate them to obtain a time-series EEG data stream. S2. Extract the time-series EEG data stream, perform filtering and denoising operations on the time-series EEG data stream based on a preset frequency band range to obtain a denoised time-series EEG signal, perform time-frequency domain transformation on the denoised time-series EEG signal to extract frequency band energy data, construct a multi-dimensional EEG feature vector, input the multi-dimensional EEG feature vector into a pre-constructed classification and analysis model, and perform feature mapping sequentially through the spatial feature aggregation layer, temporal dependency extraction layer and intention mapping fully connected layer in the classification and analysis model to obtain the current EEG control features; S3. When the current EEG control feature meets the preset trigger threshold, extract the hardware absolute sequence number range and the starting hardware timestamp from the time-series EEG data stream corresponding to the triggering current EEG control feature, and merge and encapsulate the hardware absolute sequence number range and the starting hardware timestamp as metadata with the control instructions generated based on the current EEG control feature to obtain a time-series anchored control instruction package. S4. Input the timing anchoring control instruction packet into a pre-built circular sliding window queue, extract the status snapshot containing the sequence number and check code of the historical preset number of sent instructions cached in the circular sliding window queue, save the historical control instruction code, sequence number and CRC, combine the timing anchoring control instruction packet and the status snapshot for joint assembly and redundant encoding, generate a UDP cooperative transmission message, and send the UDP cooperative transmission message to the external execution device through the UDP channel; S5. Obtain the UDP cooperative transmission message received by the external execution device, parse the hardware absolute sequence number carried in the UDP cooperative transmission message, and compare the hardware absolute sequence number with the maximum executed sequence number maintained by the local state machine. If the comparison result is that the serial numbers are not continuous, the historical instructions corresponding to the missing serial numbers are extracted based on the state snapshot parsing, the sequence is rearranged, and the time difference is calculated in combination with the starting hardware timestamp to perform instruction delay interpolation smoothing to obtain the target control instruction. The target control instruction is then sent to the external execution device to drive its execution action.

2. The method for highly reliable signal transmission in a brain-computer interface based on hardware and software collaboration according to claim 1, characterized in that: The raw, continuous EEG signals are acquired in real time. During the underlying data streaming process, a monotonically increasing hardware absolute sequence number and hardware timestamp are injected into each frame of the raw, continuous EEG signal data packet, and then encapsulated to obtain a time-series EEG data stream, as follows: S1.1 Acquire the analog EEG voltage signal collected by the front-end electrode sensor, and then convert it to a preset sampling frequency via an analog-to-digital converter. The simulated EEG voltage signal is discretized, sampled, and quantized to obtain a digital EEG data matrix containing multiple sampling points. ,in Index of discrete-time sampling points; S1.2 Process the digital EEG data matrix according to the preset data frame length. Low-level memory segmentation is performed to obtain continuous single-frame EEG data. In each frame of the single-frame EEG data When writing to the transmit buffer, an incrementing counter in the underlying hardware register is triggered to generate a hardware absolute sequence number corresponding to the current frame. The absolute serial number of the hardware satisfies a discrete recursive equation: in, This is the frame sequence number of the current data packet, and the base sequence number is defined during system power-on initialization. ; S1.3 Within the same hardware clock cycle that triggers the incrementing counter, the current cumulative oscillation cycle count value of the local hardware crystal oscillator is synchronously read. Combined with the inherent nominal frequency of the hardware crystal oscillator, the current frame sequence number is calculated. Strictly physically aligned hardware timestamps The calculation formula is as follows: in, The reference zero-point time recorded during the system synchronization handshake. For the first The accumulated crystal oscillator count value read at the frame trigger moment. The inherent nominal frequency of the hardware crystal oscillator; S1.4 According to the preset communication bus protocol bit width, the absolute serial number of the hardware is... With the hardware timestamp The kernel protocol header segment allocated to the underlying data packet, and the single frame of EEG data... The data payload segment is mapped to the underlying data packet, and a cyclic redundancy check code is appended to the end of the packet. The packets are then concatenated bit by bit to form a complete time-series data packet. Multiple time-series data packets generated consecutively are then transmitted and pushed upwards in sequence to output the time-series EEG data stream.

3. The method for highly reliable signal transmission in a brain-computer interface based on hardware and software collaboration according to claim 1, characterized in that: The current brainwave control features are obtained based on time-series EEG data stream processing, and the specific steps are as follows: S2.1 Parse the data payload segment of the time-series EEG data stream and extract the data containing... The original multichannel EEG sequence for each lead channel, targeting any lead channel The EEG sequence of this channel is filtered and denoised using a preset infinite impulse response bandpass filter to obtain the denoised temporal EEG signal. The corresponding time-domain linear constant-coefficient difference equation is: in, For discrete-time sampling point index, For lead channel numbering, For the first segment extracted from the payload segment The current input sample value for the channel. and These are the orders of the forward and feedback filters, respectively. For the first Forward feed coefficient, For the first The passband cutoff range of the bandpass filter is set according to the preset EEG characteristic frequency band range parameter, which is the first-order feedback coefficient. S2.2 Set the sliding time window length to And the window movement step length is The denoised temporal EEG signals of each lead channel Simultaneously perform short-time Fourier transform and calculate the target frequency band. Cumulative frequency band energy data within The calculation formula is as follows: in, This is the index of the current window frame of the sliding time window. For discrete frequency indexing, Hamming weighted window function to suppress spectral leakage, and The indexes of the lower and upper frequency limits of the preset frequency band range in the discrete spectrum are respectively used to index the current window frame. All The accumulated frequency band energy data of each channel are spliced ​​together according to the lead space topology to construct the multidimensional EEG feature vector. ; S2.3 The multidimensional EEG feature vector Input the spatial feature aggregation layer of the classification parsing model, and pass it through a pre-trained spatial convolution weight matrix. Nonlinear feature aggregation of spatial dimensions between channels is performed to reduce dimensionality and obtain spatial fusion features. : The spatial fusion features are then... The temporal dependency extraction layer of the classification parsing model is sequentially input, combined with the hidden state of the previous window frame. Recursively compute the execution temporal context to obtain fused intent features that include dependencies on historical action sequences. : in, , , These are the spatial convolution weight matrix, the temporal input weight matrix, and the temporal recursive weight matrix, respectively. and For the corresponding bias vector, and All are non-linear activation functions; S2.4 The fusion intent features The input intent mapping fully connected layer is used to calculate the state probability distribution, and the probability vector of the current state belonging to each preset intent category is output through a normalized exponential function. and by taking the maximum probability index function from the probability vector Extract the corresponding classification labels as the final current EEG control features. : in, This is the mapping matrix for the fully connected layer. For the bias vector of the fully connected layer, This represents the total number of preset categories of external device control actions. The probability vector The first in A probability element, the current brainwave control feature Used to characterize the current discrete action intent of the target user.

4. The method for highly reliable signal transmission in a brain-computer interface based on hardware and software collaboration according to claim 3, characterized in that: The pre-built classification parsing model includes the following steps in its construction and training process: Acquire historical continuous EEG data carrying the true intention action tags of external devices, and perform filtering, noise reduction and time-frequency domain feature extraction operations on the historical continuous EEG data to obtain a historical multidimensional EEG feature vector sequence. The true intention action labels are converted into true probability distribution vectors in one-hot encoded format. The historical multidimensional EEG feature vector sequence is then aligned with the corresponding true probability distribution vectors to construct a training sample set. in For sample time step index, For the first A historical multidimensional EEG feature vector at each time step, This represents the corresponding true probability distribution vector. The total number of training samples; An initial classification and parsing model is constructed. The spatial convolutional weight matrix and bias vector of the spatial feature aggregation layer, the temporal input weight matrix, the temporal recursive weight matrix and bias vector of the temporal dependency extraction layer, and the fully connected layer mapping matrix and bias vector of the intent mapping fully connected layer are randomly initialized using a normal distribution. The historical multidimensional EEG feature vector sequence from the training sample set is then used. The initial classification model is input according to a preset batch size, and then sequentially passes through the dimensionality reduction operation of the spatial feature aggregation layer, the historical state recursive operation of the temporal dependency extraction layer, and the normalized exponential mapping of the intent mapping fully connected layer, outputting the index of the sample time step. The corresponding predicted probability distribution vector ; Extract the predicted probability distribution vector and the true probability distribution vector corresponding to the training sample set. The cross-entropy cost function is used to calculate the global classification loss value of the forward propagation of the current batch. The calculation formula is as follows: in, This refers to the number of samples in a single training iteration. For the total number of intended categories, For the first batch The th sample in the true probability distribution vector One element, For the first batch The th sample in the predicted probability distribution vector One element; Based on the calculated global classification loss value The global classification loss value is calculated using a backpropagation algorithm based on the chain rule. The loss gradient of each weight matrix and bias vector in the initial classification parsing model; An adaptive moment estimation optimization algorithm is used to iteratively update the weight matrix and bias vector for each term based on the loss gradient. The weight parameter update formula is as follows: in, Represents any weight matrix or bias vector to be updated. For the current iteration round, The preset base learning rate, This is the first-order momentum estimate after bias correction for the current iteration. This is the second-order momentum estimate after bias correction for the current iteration round. To prevent the smoothing constant from having a denominator of zero; Repeat the above training steps until the global classification loss value is reached. If the value is less than the preset convergence threshold or the maximum number of iterations is reached, stop updating the parameters and solidify all the weight matrices and bias vectors, and output the pre-built classification parsing model that has been trained and converged.

5. A highly reliable signal transmission method for brain-computer interfaces based on hardware and software collaboration according to claim 3, characterized in that: Extract the hardware absolute sequence number range and the starting hardware timestamp from the time-series EEG data stream corresponding to the triggering current EEG control feature. Then, merge and encapsulate the hardware absolute sequence number range and the starting hardware timestamp as metadata with the control instructions generated based on the current EEG control feature to obtain a time-series anchored control instruction package, specifically including: S3.1 Based on the probability vector Extract the maximum classification probability value. The maximum classification probability value Compared with the preset confidence threshold of intent Perform numerical comparison, and when a judgment is made At that time, confirm the current brainwave control characteristics. If the preset trigger threshold condition is met, and the preset device action command mapping dictionary is extracted, the current EEG control features are mapped according to the unique mapping relationship. Transformed into specific control command encoding for the external device ; S3.2 Obtain the features used to extract the current EEG control features Current sliding time window index Combined with the preset sliding time window length Window moving step length and the number of fixed sampling points contained in a single frame data packet in the time-series EEG data stream. Calculate the hardware absolute sequence number range that triggers the current EEG control feature. The calculation formula is as follows: in, The reference sequence number is set during system power-on initialization. This is the floor function. The starting sequence number of the interval. This is the end sequence number of the interval; Synchronously query the time-series EEG data stream and extract the sequence number corresponding to the starting sequence number of the interval. The hardware timestamp recorded in the kernel protocol header segment of the corresponding data packet is used as the starting hardware timestamp. ; S3.3 Calculate the hardware absolute serial number range and the starting hardware timestamp The data is divided into time-series metadata fields, and these time-series metadata fields are concatenated to the specific control instruction encoding. The head position is determined, and a check bit calculated based on a preset generator polynomial is added to the tail. These are then combined and encapsulated to form the timing anchoring control instruction package. The data structure expression of the timing anchoring control instruction packet is as follows: in, This represents a concatenation of consecutive bit streams of data fields in memory. This is a cyclic redundancy check sequence calculated for the entire current instruction packet.

6. The method for highly reliable signal transmission in a brain-computer interface based on hardware and software collaboration according to claim 1, characterized in that: The timing anchoring control command packet is input into a pre-built circular sliding window queue. A state snapshot containing the sequence number and checksum of a preset number of historically transmitted commands is extracted from the circular sliding window queue. The timing anchoring control command packet and the state snapshot are jointly assembled and redundantly encoded to generate a UDP cooperative transmission message. The UDP cooperative transmission message is then sent to an external execution device through a UDP channel, as detailed below: S4.1 The timing anchoring control instruction package Push into a pre-built circular sliding window queue The position of the tail pointer when the circular sliding window queue The current storage length exceeds the preset maximum window capacity. At that time, the earliest historical command packet at the head of the queue will pop up; From the circular sliding window queue Extract the preceding order of the current tail pointer. The state snapshot is constructed by extracting the interval start sequence number and cyclic redundancy check sequence corresponding to each historical frame from the historical frames. Its mathematical expression is: in, This represents a structure array with a fixed length. This represents a concatenation of consecutive bit streams of data fields in memory. The preset number of frames is the preset historical frame size and meets the following requirements. , and The first The starting sequence number and check code of the interval corresponding to the frame history instruction; S4.2 The timing anchoring control instruction package With the aforementioned state snapshot Perform bitwise concatenation to obtain a combined data block. Extract the preset forward error correction coding matrix Through the forward error correction coding matrix For the joint data block Perform Galois field multiplication redundancy coding operations to generate corresponding redundancy check blocks. and the joint data block With the redundant check block Merging and assembling to generate transmission payload The calculation formula is as follows: in, Indicates the presupposed Galois domain Finite field matrix multiplication, To generate the linear generator matrix of the system error-correcting code, A redundant check byte stream used to combat packet loss during network transmission; S4.3 According to the underlying standard of the connectionless User Datagram Protocol, the transmitted payload is... The UDP cooperative transmission message is encapsulated by appending a transport layer header consisting of the source port and the destination port, a network layer header containing the IP address of the destination external execution device, and a data link layer physical frame header in sequence. Its data structure expression is: in, , , These are the standard communication header segments for the Media Access Control layer, Internet Protocol layer, and User Datagram Protocol layer, respectively; finally, by calling the underlying network interface card hardware driver, the continuously generated UDP cooperative transmission messages are... The data is sent asynchronously to the external execution device via a UDP channel.

7. A highly reliable signal transmission method for brain-computer interfaces based on hardware and software collaboration as described in claim 6, characterized in that: The external execution devices include embodied robots, aircraft, detectors, immersive gaming / XR devices, and virtual engine environments.

8. The method for highly reliable signal transmission in a brain-computer interface based on hardware and software collaboration according to claim 1, characterized in that: In step S5, the generation of the target control command is as follows: S5.1 Obtain the UDP cooperative transmission message received by the external execution device at the transport layer, perform reverse data unpacking on the UDP cooperative transmission message, and extract the current hardware absolute sequence number range carried in the payload of the current message. Current starting hardware timestamp and current control commands Simultaneously, it reads the maximum executed sequence number maintained in the local receiving state machine memory of the external execution device. The starting serial number in the current hardware absolute serial number range With the maximum sequence number already executed Perform numerical calculations to determine the difference in serial numbers. ; S5.2 If the difference in the serial numbers is determined If the comparison result indicates that the sequence number is discontinuous, the redundant appended state snapshot in the UDP cooperative transmission message is extracted. The state snapshot contains historical data. Frame control commands and corresponding sequence number mapping table ; Based on the set of missing sequence numbers In the mapping table A matching search is performed to extract the missing historical instructions, and the extracted historical instructions are compared with the current control instructions. The memory addresses are rearranged according to the monotonically increasing absolute sequence number to construct a sequentially continuous queue of instructions to be executed. ; S5.3 For the queue of instructions to be executed The instruction in the command retrieves the current system rendering clock of the external execution device. Combined with the current starting hardware timestamp Calculate the time difference of end-to-end transmission scheduling of instructions. An exponential decay interpolation algorithm is used based on the time difference. The formula for smoothing compensation of command control parameters is as follows: in, The target control command is after smooth compensation. For the queue of instructions to be executed The original command parameters for the current dequeue. These are the historical execution instruction parameters cached by the external execution device in the previous machine response cycle. The adaptive interpolation weights are dynamically calculated based on the time difference. The preset external execution device smoothing attenuation constant; S5.4 The target control command obtained after the smoothing compensation process. The instructions are sequentially sent to the underlying driver controller of the external execution device to drive the external execution device to execute the corresponding continuous actions. After all actions in the current instruction queue have been executed, the maximum executed sequence number maintained by the local receiving state machine is transferred. Synchronously updated to the termination serial number within the current hardware absolute serial number range. This completes the state iteration closed loop of the single-interaction control state machine.

9. A high-reliability signal transmission system for a brain-computer interface based on hardware and software collaboration, used to implement the high-reliability signal transmission method for a brain-computer interface based on hardware and software collaboration as described in any one of claims 1 to 8, characterized in that, include: The data acquisition and timing encapsulation module is used to acquire raw continuous EEG signals in real time. During the underlying data push process, it injects a monotonically increasing hardware absolute sequence number and hardware timestamp into each frame of the raw continuous EEG signal data packet, and encapsulates it to obtain a timing EEG data stream. The signal preprocessing and feature parsing module is used to extract the time-series EEG data stream, perform filtering and denoising operations on the time-series EEG data stream based on a preset frequency band range to obtain a denoised time-series EEG signal, perform time-frequency domain transformation on the denoised time-series EEG signal to extract frequency band energy data, construct a multi-dimensional EEG feature vector, input the multi-dimensional EEG feature vector into a pre-constructed classification parsing model, and perform feature mapping sequentially through the spatial feature aggregation layer, temporal dependency extraction layer and intention mapping fully connected layer in the classification parsing model to obtain the current EEG control features; The timing anchoring instruction encapsulation module is used to extract the hardware absolute sequence number range and the starting hardware timestamp from the timing EEG data stream corresponding to the current EEG control feature when the current EEG control feature meets the preset trigger threshold, and to merge and encapsulate the hardware absolute sequence number range and the starting hardware timestamp as metadata with the control instruction generated based on the current EEG control feature to obtain a timing anchoring control instruction package. The redundant snapshot cooperative transmission module is used to input the timing anchoring control instruction packet into a pre-built circular sliding window queue, extract the status snapshots containing the sequence numbers and check codes of the historical preset number of sent instructions cached in the circular sliding window queue, save the historical control instruction codes, sequence numbers and CRCs, jointly assemble the timing anchoring control instruction packet and the status snapshots and perform redundant encoding to generate a UDP cooperative transmission message, and send the UDP cooperative transmission message to an external execution device through a UDP channel; The receiving rearrangement and smooth execution module is used to obtain the UDP cooperative transmission message received by the external execution device, parse the hardware absolute sequence number carried in the UDP cooperative transmission message, and compare the hardware absolute sequence number with the maximum executed sequence number maintained by the local state machine. If the comparison result is that the serial numbers are not continuous, the historical instructions corresponding to the missing serial numbers are extracted based on the state snapshot parsing, the sequence is rearranged, and the time difference is calculated in combination with the starting hardware timestamp to perform instruction delay interpolation smoothing to obtain the target control instruction. The target control instruction is then sent to the external execution device to drive its execution action.

10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, it employs the high-reliability signal transmission method for brain-computer interface based on hardware and software collaboration as described in any one of claims 1 to 8.

Images