Training of target discrimination model of brain-computer interface, and multi-type target discrimination method and device

Abstract

The application relates to the technical field of brain-computer interfaces, and provides a training method and device for a target discrimination model of a brain-computer interface and a multi-type target discrimination method and device. The method comprises the following steps: inputting multi-modal physiological signal sample data into an initial decoding model to obtain task common characteristics and individual physiological characteristics; determining a target classification loss based on the task common characteristics and a task target label; determining an orthogonal constraint loss based on the orthogonality of the task common characteristics and the individual physiological characteristics in a feature space; and updating model parameters of the model based on the target classification and the orthogonal constraint loss. By introducing the orthogonal constraint loss, the task common characteristics and the individual physiological characteristics are forced to be separated, so that the cross-user common characteristics directly related to task target discrimination and the individual physiological characteristics reflecting the unique physiological background of a user are effectively decoupled in the model training stage, the model can learn task discrimination characteristics with higher purity which are independent of individual differences, and the decoding effect in a cross-user application scenario is ensured.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and in particular to a method and apparatus for training a target discrimination model for a brain-computer interface and for discriminating multiple types of targets. Background Technology

[0002] Brain-computer interface (BCI) technology breaks through the limitations of traditional interaction methods that rely on peripheral nerves and muscle tissue by establishing a direct communication channel between the human brain and external devices. BCI systems typically acquire neurophysiological signals from the brain during specific cognitive tasks, such as electroencephalograms (EEGs), using non-invasive or invasive sensors. Machine learning and pattern recognition algorithms are then used to decode the neural intentions within these signals, ultimately converting them into control commands. In recent years, with the deep integration of neuroscience, sensor technology, and artificial intelligence algorithms, BCI technology has demonstrated enormous application potential and strategic value in areas such as medical rehabilitation (e.g., motor function reconstruction, diagnosis of consciousness disorders), specialized human-computer interaction (e.g., intelligent cockpits, exoskeleton control), and enhancement of human cognitive function.

[0003] Brain-computer interfaces based on Rapid Serial Visual Presentation (RSVP) (RSVP-BCI) have significant advantages in high-speed visual target detection tasks. The RSVP paradigm presents visual stimuli, such as image sequences, continuously at the same spatial location with an extremely high frame rate. When a subject identifies a pre-defined "target" within a rapidly switching visual stream, the brain generates specific event-related potentials (ERPs), primarily manifested as N2pc components evoked in the visual cortex and P300 slow waves distributed in the parietal lobe. RSVP-BCI cleverly combines the powerful higher-level visual cognitive abilities of the human brain with the rapid processing capabilities of computers, enabling users to process massive amounts of visual information in a short time. It demonstrates extremely high application efficiency in high-throughput target retrieval scenarios such as wide-area security monitoring and drone reconnaissance image screening. With the development of deep learning technology, the decoding method of RSVP-BCI has undergone technological iterations from traditional spatial filtering to convolutional neural networks. Current decoding algorithms strive to eliminate reliance on cumbersome intra-individual calibration data, evolving to a "zero-calibration" stage for single-class targets in image sequences. Zero-calibration models learn generalized representations of ERPs on multi-source subject datasets, enabling new users to directly perform target retrieval tasks without prior collection of specific calibration data, significantly improving the plug-and-play nature and deployment efficiency of the RSVP-BCI system.

[0004] However, for large-scale applications in complex and dynamic real-world scenarios, existing RSVP-BCI technology still faces significant technical bottlenecks: First, existing stimulus paradigms are mostly limited to image sequences and single-class target retrieval; second, static "zero-calibration" pre-trained models cannot predict and overcome data distribution shifts in unknown new subjects, and lack adaptive and compensatory mechanisms to cope with sudden modal loss such as electrode detachment or eye movement off-target during long-term online operation, making it difficult to maintain the recognition accuracy of the system in complex and non-steady-state environments in the long term. Summary of the Invention

[0005] This invention provides a brain-computer interface target discrimination model training method and device for multi-type target discrimination, which solves the defects of existing stimulation paradigms that are limited to single-class target retrieval in image sequences, cannot adapt to multi-class detection in continuous dynamic video streams, and static zero-calibration pre-trained models are difficult to overcome the distribution bias of new subject data and lack adaptive decoupling and compensation mechanisms when modality is missing, resulting in the system's recognition accuracy being unable to be maintained for a long time in complex non-steady-state environments.

[0006] This invention provides a method for training a target discrimination model for a brain-computer interface. The target discrimination model is obtained by iteratively executing the following steps until a preset iteration termination condition is met: Acquire multimodal physiological signal sample data and corresponding task target labels; The multimodal physiological signal sample data is input into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model. Based on the common features of the tasks and the task target labels, the target classification loss is determined; Based on the orthogonality of the common features of the tasks and the individual physiological features in the feature space, the orthogonal constraint loss is determined. Based on the target classification loss and the orthogonal constraint loss, the target loss is determined, and the model parameters of the initial decoding model are updated based on the target loss.

[0007] According to the training method of a target discrimination model for a brain-computer interface provided by the present invention, the step of determining the target loss based on the target classification loss and the orthogonal constraint loss includes: Obtain the user tags corresponding to the multimodal physiological signal sample data; Based on the individual physiological characteristics and the user tags, determine the user classification loss; The target loss is determined based on the target classification loss, the orthogonal constraint loss, and the user classification loss.

[0008] According to the training method of a target discrimination model for a brain-computer interface provided by the present invention, the step of acquiring multimodal physiological signal sample data includes: Acquire raw physiological signal data including multiple of the task target categories; Extract the real physiological signal data corresponding to the target category with sparse frequency in the original physiological signal data as seed data; The seed data is input into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model. The original physiological signal data is combined with the synthesized physiological signal data to obtain the multimodal physiological signal sample data.

[0009] According to a training method for a target discrimination model of a brain-computer interface provided by the present invention, the step of inputting the seed data into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model includes: The conditional diffusion generation model determines the macroscopic structure prior condition matrix based on the low-frequency amplitude envelope extracted from the seed data. The time-frequency energy distribution and instantaneous phase are extracted from the seed data as microscopic detail constraint condition tensors. Based on the macroscopic structure prior condition matrix, a first-stage inverse denoising process is performed on Gaussian noise to generate a preliminary synthesized signal with a global temporal trend. Based on the preliminary synthesized signal and the microscopic detail constraint condition tensor, a second-stage inverse denoising process is performed to progressively reconstruct and generate the synthesized physiological signal data.

[0010] According to a training method for a target discrimination model of a brain-computer interface provided by the present invention, the training steps of the conditional diffusion generation model include: Determine the initial conditional diffusion generation model; the initial conditional diffusion generation model includes an initial first-level network and an initial second-level network; Obtain sample seed data, and gradually inject Gaussian noise into the sample seed data to obtain multi-step noisy data; The macroscopic structure prior condition matrix corresponding to the sample seed data and the noisy data corresponding to the first stage time step in the multi-step noisy data are input into the initial first-level network to obtain the first noise prediction result and the preliminary synthesized signal output by the initial first-level network. Based on the difference between the first noise prediction result and the Gaussian noise, a first training loss is determined, and the parameters of the initial first-level network are iterated based on the first training loss to obtain a trained first-level network. Obtain the sample task target category label corresponding to the sample seed data; The micro-detail constraint tensor corresponding to the sample seed data, the sample task target category label, the preliminary synthesized signal, and the noisy data corresponding to the second stage time step in the multi-step noisy data are input into the initial second-level network to obtain the second noise prediction result output by the initial second-level network. Based on the difference between the second noise prediction result and the Gaussian noise, a second training loss is determined, and the parameters of the initial second-level network are iterated based on the second training loss to obtain a trained second-level network. Based on the trained first-level network and the trained second-level network, the conditional diffusion generation model is determined.

[0011] This invention also provides a method for identifying multiple types of targets, comprising the following steps: Acquire multimodal physiological data to be identified; the multimodal physiological data to be identified includes video data of the target to be identified, EEG data corresponding to the target to be identified, and eye movement data corresponding to the target to be identified; The multimodal physiological data to be discriminated is input into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; The target discrimination model is obtained by performing the training method based on the target discrimination model of the brain-computer interface described above.

[0012] According to a multi-type target discrimination method provided by the present invention, the step of inputting the multimodal physiological data to be discriminated into a target discrimination model to obtain the target discrimination prediction result output by the target discrimination model includes: Obtain the real-time signal quality of all modalities in the multimodal physiological data to be discriminated; The real-time signal quality is input into the target discrimination model. In response to determining that the real-time signal quality of the target modality among all modalities is lower than a preset robustness threshold, the target modality is identified as a missing modality. Based on the signal of at least one intact modality other than the missing modality in the physiological data of the multimodalities to be discriminated, the missing modality representation corresponding to the missing modality is reconstructed in the latent space of the parallel encoder in the target discrimination model. Based on the missing modality representation, a compensating multimodal feature is determined, and the target discrimination prediction result is output based on the compensating multimodal feature.

[0013] According to a multi-type target discrimination method provided by the present invention, the target discrimination model further includes a dynamic gating network, and a common expert module, an individual representation expert module, and a modal complementarity expert module connected to the dynamic gating network; The step of inputting the multimodal physiological data to be discriminated into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model further includes: The multimodal physiological data to be discriminated is input into the latent space of the parallel encoder for encoding to obtain the latent space representation vector; The latent space representation vector is input into the dynamic gating network to obtain the first routing weight corresponding to the common expert module, the second routing weight corresponding to the individual representation expert module, and the third routing weight corresponding to the modal complementarity expert module output by the dynamic gating network. Based on the first routing weight, the latent space representation vector is input to the commonality expert module to obtain the cross-user task commonality prediction features output by the commonality expert module; Based on the second routing weight, the latent space representation vector is input to the individual representation expert module to obtain the individual physiological prediction features of the current user output by the individual representation expert module; Based on the third routing weight, the latent space representation vector is input to the modal complementarity expert module to obtain the interaction information between multimodal signals in the multimodal physiological data to be discriminated, which is output by the modal complementarity expert module. Based on the common prediction features of the task, the individual physiological prediction features, and the interaction information, the target discrimination prediction result is determined.

[0014] The present invention also provides a training device for a target discrimination model of a brain-computer interface, comprising the following units: The first acquisition unit is used to acquire multimodal physiological signal sample data and task target labels corresponding to the multimodal physiological signal sample data; The first input unit is used to input the multimodal physiological signal sample data into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model. The first determining unit is used to determine the target classification loss based on the common features of the task and the target label of the task; The second determining unit is used to determine the orthogonal constraint loss based on the orthogonality of the common features of the task and the individual physiological features in the feature space; The update unit is used to determine the target loss based on the target classification loss and the orthogonal constraint loss, and update the model parameters of the initial decoding model based on the target loss.

[0015] The present invention also provides a multi-type target discrimination device, comprising the following units: The second acquisition unit is used to acquire the multimodal physiological data to be identified; the multimodal physiological data to be identified includes video data of the target to be identified, EEG data corresponding to the target to be identified, and eye movement data corresponding to the target to be identified. The second input unit is used as an input unit to input the multimodal physiological data to be discriminated into the target discrimination model and obtain the target discrimination prediction result output by the target discrimination model. The target discrimination model is obtained by performing the training method based on the target discrimination model of the brain-computer interface described above.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a training method for a target discrimination model of a brain-computer interface as described above, or implements a multi-type target discrimination method as described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a training method for a target discrimination model of a brain-computer interface as described above, or implements a multi-type target discrimination method as described above.

[0018] The present invention also provides a computer program product, comprising a computer program that, when executed by a processor, implements a training method for a target discrimination model of a brain-computer interface as described above, or implements a multi-type target discrimination method as described above.

[0019] The present invention provides a brain-computer interface target discrimination model training method and apparatus for multi-type target discrimination, which acquires multimodal physiological signal sample data and corresponding task target labels; inputs the multimodal physiological signal sample data into an initial decoding model including a common feature expert module and an individual representation expert module to output task common features and individual physiological features respectively; determines the target classification loss based on the task common features and task target labels; determines the orthogonal constraint loss based on the orthogonality of the task common features and individual physiological features in the feature space; determines the target loss based on the target classification loss and the orthogonal constraint loss together, and updates the model parameters of the initial decoding model. This invention introduces orthogonal constraint loss during model training to forcibly separate common task features from individual physiological features. This effectively decouples cross-user common features directly related to task target discrimination from individual physiological features reflecting user-specific physiological backgrounds during the model training phase. Consequently, the target discrimination model can learn higher-purity task discrimination features independent of individual differences, improving the generalization ability and robustness of the target discrimination model when faced with data distribution shifts in unknown new subjects. This solves the problem that the model cannot be applied to multi-class target detection tasks in continuous dynamic video streams due to stimulus paradigm limitations and interference from individual differences, ensuring the decoding performance of the brain-computer interface system in cross-user application scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the training method of the target discrimination model for the brain-computer interface provided by the present invention.

[0022] Figure 2 This is a flowchart illustrating the multi-type target discrimination method provided by the present invention.

[0023] Figure 3 This is a schematic diagram of the structure of the training device for the target discrimination model of the brain-computer interface provided by the present invention.

[0024] Figure 4 This is a schematic diagram of the structure of the multi-type target discrimination device provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and that the objects distinguished by "first," "second," etc., are generally of the same class.

[0028] In related technologies, current brain-computer interface (BCI) technologies, especially decoding models for the Rapid Serial Visual Presentation (RSVP) paradigm, generally face performance bottlenecks due to individual physiological differences when applied across users. Different users exhibit significant differences in their EEG signal baselines, noise levels, and response patterns to visual stimuli. This makes it difficult to directly generalize a model trained on one user to new, unknown users, typically requiring a cumbersome intra-individual calibration process. This severely limits the practicality and deployment efficiency of BCI systems.

[0029] This invention designs a novel model training framework that proactively decouples common features related to task target discrimination from individual user-specific physiological features during the training phase, thereby learning a more generalizable target discrimination model. This target discrimination model can extract robust, cross-user task-related neural representations, reducing reliance on individual-specific physiological signal patterns.

[0030] This invention provides a training method for a target discrimination model of a brain-computer interface. This method is applied to a brain-computer interface system, which typically includes a data acquisition module for collecting user physiological signals, a preprocessing module for processing the signals, and a target discrimination model for decoding user intentions from the signals. Figure 1 This is a flowchart illustrating the training method of the target discrimination model for the brain-computer interface provided by the present invention, as shown below. Figure 1 As shown, the target discrimination model is obtained by iteratively executing the following steps until a preset iteration termination condition is met. The preset iteration termination condition may include the performance index of the initial decoding model on the reserved validation set reaching a preset threshold or no longer significantly improving within multiple consecutive training cycles; or the total number of training rounds reaching a preset upper limit. This embodiment of the invention does not specifically limit this.

[0031] The method includes the following steps: Step 110: Obtain multimodal physiological signal sample data and the task target label corresponding to the multimodal physiological signal sample data.

[0032] Specifically, firstly, multimodal physiological signal sample data and corresponding task target labels can be obtained.

[0033] Multimodal physiological signal sample data refers to one or more sets of data reflecting a user's physiological state when performing a specific cognitive task. Multimodal means the data originates from multiple different types of sensors or signal sources, capable of characterizing the user's physiological response from different dimensions. In a specific application scenario, multimodal physiological signal sample data may include electroencephalogram (EEG) signal data and eye-tracking signal data. For example, EEG signal data can be acquired through electrodes worn on the user's scalp, reflecting the neural electrical activity of the cerebral cortex; eye-tracking signal data can be acquired through an eye tracker, including information such as the user's fixation point coordinates, pupil diameter changes, and saccade trajectories, reflecting the user's attention and visual processing in visual search tasks. Furthermore, in other embodiments, multimodal physiological signal sample data may also include other types of physiological signals, such as electrocardiogram (ECG) signals and electromyographic (EMG) signals; this embodiment of the invention does not specifically limit this.

[0034] Acquiring multimodal physiological signal sample data is typically based on a pre-defined experimental paradigm. For example, a rapid sequence visual presentation paradigm can be used, where a series of visual stimuli, such as images, characters, or video frames, are rapidly presented to the user. A small portion of these visual stimuli are the "targets" the user needs to identify, while the majority are "non-target" distractors. When the user identifies the target, their brain generates specific event-related potentials, such as the P300 component, and their eye movement patterns may also exhibit different characteristics compared to when identifying non-targets. By synchronously recording the user's multimodal physiological signals for a short period before and after each visual stimulus using experimental equipment, and segmenting the continuous signal stream—for example, taking the stimulus presentation time as zero and extracting signal segments from -200 milliseconds to 1000 milliseconds before and after, i.e., signal segments of [-200, 1000] milliseconds—a multimodal physiological signal sample data can be constructed.

[0035] Here, the task target label is used to identify whether each multimodal physiological signal sample data corresponds to a user's physiological response to a "target" stimulus. The task target label includes at least two different semantic categories of target identifiers, corresponding to different categories of targets to be detected in the continuous dynamic video stream. The task target label is a truth label in supervised learning. For example, binary encoding can be used; when the multimodal physiological signal sample data is acquired when a "target" stimulus is presented, its corresponding task target label can be set to "1" or "True"; when the multimodal physiological signal sample data is acquired when a "non-target" stimulus is presented, its corresponding task target label can be set to "0" or "False".

[0036] Step 120: Input the multimodal physiological signal sample data into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model.

[0037] Specifically, after obtaining multimodal physiological signal sample data, the multimodal physiological signal sample data can be input into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model.

[0038] The parameters of the initial decoding model can be preset or randomly generated; this embodiment of the invention does not impose specific limitations on this.

[0039] Here, the structural feature of the initial decoding model is that it includes at least two functionally specialized sub-network modules, namely the commonality expert module and the individual representation expert module.

[0040] In some more complex implementations, the initial decoding model may also include a modal complementarity expert. It is used to calculate cross-modal interaction information between multi-source physiological signals.

[0041] In a specific embodiment, let the input multimodal physiological signal sample data be... ,in For electroencephalogram (EEG) signal data, For eye movement signal data, Optional, physiologically synchronized video semantic features. The brain-computer interface system utilizes parallel encoder branches to acquire the representation vectors of multimodal physiological signal sample data in the hidden space. Heterogeneous features are mapped to a unified, decoupled manifold space through cascaded operations or self-attention mechanisms. Subsequently, a dynamic gating network is constructed. Real-time calculation of the contribution weights of input features among expert layers: ; in, Indicates contribution weight. This represents the representation vector of multimodal physiological signal sample data in the hidden space. This indicates the weight of the common expert module. The weights of the individual representation expert modules are represented. This represents the weight of the modal complementarity expert module. This contribution weight guides the feature flow into three parallel expert module layers.

[0042] It should be noted that the model structure of the common expert module, the individual representation expert module, and the modality complementarity expert module can be a cascaded multilayer convolutional neural network (CNN), a deep neural network (DNN), or a combination of CNN and DNN, etc. The embodiments of the present invention do not specifically limit this.

[0043] In one possible implementation, the initial decoding model may first include a parallel encoder, for example, using a convolutional neural network to process EEG signal data and eye movement signal data separately, extract shallow feature representations of EEG signal data and eye movement signal data, and then input these shallow feature representations into subsequent expert module layers.

[0044] The commonality expert module is designed to learn and extract features that remain relatively stable and consistent across different users and experimental time periods, and are directly related to completing the target discrimination task. These commonality features reflect core task evoked information independent of individual differences, aiming to capture core neural response patterns independent of individual variations. Taking the RSVP task as an example, different participants all elicited the P300 component in response to the target stimulus. Although the specific amplitude and morphology vary among individuals, the evoked pattern and spatiotemporal distribution exhibit cross-participant stability.

[0045] The individual representation expert module and the commonality expert module are set up in parallel. The purpose of the individual representation expert module is to adaptively learn and extract individualized signal patterns that reflect the user's unique physiological background, inherent signal noise, and task-independent but stable signals. Here, individual physiological features are used to capture the physiological bias and non-stationary noise floor of the current specific user.

[0046] Step 130: Determine the target classification loss based on the common features of the task and the target label of the task.

[0047] Specifically, the target classification loss can be determined based on common task features and task target labels. Here, the target classification loss is a loss function used to measure the model's classification performance. The process of determining the target classification loss is as follows: common task features are input into a classifier, which outputs a predicted probability distribution. This distribution represents the probability that the multimodal physiological signal sample data corresponds to a target and non-target. Then, the predicted probability distribution is compared with the true task target labels of the multimodal physiological signal sample data, for example, their one-hot encoded form.

[0048] In one specific embodiment, the target classification loss can be calculated using cross-entropy loss. By minimizing the target classification loss, the commonality expert module can be trained to have highly class discriminative output features for the common tasks.

[0049] It is understandable that the greater the difference between the common features of the task and the target label, the greater the target classification loss; the smaller the difference between the common features of the task and the target label, the smaller the target classification loss.

[0050] Step 140: Determine the orthogonal constraint loss based on the orthogonality of the common features of the task and the individual physiological features in the feature space.

[0051] Specifically, an orthogonal constraint loss is determined based on the orthogonality of common task features and individual physiological features in the feature space. The purpose of this step is to force the common task features and individual physiological features learned by the initial decoding model to be mathematically decoupled and unrelated. This constraint mechanism aims to guide the initial decoding model to effectively separate task-related information from individual-specific information during training.

[0052] Here, the orthogonal constraint loss is a loss function used to penalize the correlation between common task features and individual physiological characteristics. The closer the common task features and individual physiological characteristics are to orthogonality, the smaller the orthogonal constraint loss. The orthogonal constraint loss is determined by calculating the common task features. With individual physiological characteristics The similarity between feature vectors is calculated, and a loss function is constructed to minimize it. In a preferred embodiment, cosine similarity can be used to measure the correlation between feature vectors, thus the orthogonal constraint loss... It can be determined by the following formula: ; in, This represents the orthogonal constraint loss. Indicates common characteristics of tasks. Indicates individual physiological characteristics, Represents the vector dot product. This represents the summation or averaging of the loss over all multimodal physiological signal sample data in a batch. By minimizing the orthogonal constraint loss, it is possible to ensure that the two sets of feature vectors—the common features of the task and the individual physiological features—are mutually perpendicular in the feature space.

[0053] Step 150: Determine the target loss based on the target classification loss and the orthogonal constraint loss, and update the model parameters of the initial decoding model based on the target loss.

[0054] Specifically, after obtaining the target classification loss and the orthogonality constraint loss, the target loss can be determined based on the target classification loss and the orthogonality constraint loss, and the model parameters of the initial decoding model can be updated based on the target loss.

[0055] The target loss is the final loss function used to guide the training of the entire initial decoding model. In one embodiment, the target loss can be determined by weighted summation, as shown in the following formula: ; in, Indicates target loss. This represents the orthogonal constraint loss. Indicates the weighting coefficient. This represents the target classification loss.

[0056] After determining the target loss, the backpropagation algorithm is used to calculate the gradient of the target loss with respect to all trainable parameters in the initial decoding model. Subsequently, an optimizer is used to update all trainable parameters of the model based on the calculated gradient.

[0057] Finally, the initial decoded model with updated model parameters is used as the target discrimination model.

[0058] The method provided in this invention involves: acquiring multimodal physiological signal sample data and corresponding task target labels; inputting the multimodal physiological signal sample data into an initial decoding model including a common feature expert module and an individual representation expert module to output task common features and individual physiological features respectively; determining the target classification loss based on the task common features and task target labels; determining the orthogonal constraint loss based on the orthogonality of the task common features and individual physiological features in the feature space; determining the target loss based on the target classification loss and the orthogonal constraint loss together, and updating the model parameters of the initial decoding model. This invention introduces orthogonal constraint loss during model training to forcibly separate common task features from individual physiological features. This effectively decouples cross-user common features directly related to task target discrimination from individual physiological features reflecting user-specific physiological backgrounds during the model training phase. Consequently, the target discrimination model can learn higher-purity task discrimination features independent of individual differences, improving the generalization ability and robustness of the target discrimination model when faced with data distribution shifts in unknown new subjects. This solves the problem that the model cannot be applied to multi-class target detection tasks in continuous dynamic video streams due to stimulus paradigm limitations and interference from individual differences, ensuring the decoding performance of the brain-computer interface system in cross-user application scenarios.

[0059] Based on the above embodiments, step 150, determining the target loss based on the target classification loss and the orthogonal constraint loss, includes: Step 150-1: Obtain the user tags corresponding to the multimodal physiological signal sample data; Step 150-2: Determine the user classification loss based on the individual physiological characteristics and the user tags; Step 150-3: Determine the target loss based on the target classification loss, the orthogonal constraint loss, and the user classification loss.

[0060] Specifically, the first step is to obtain user tags corresponding to the multimodal physiological signal sample data. These user tags uniquely identify the user who provided the current multimodal physiological signal sample data. In a training set including data from multiple users, each user is assigned a unique identifier, such as "User 01," "User 02," etc. User tags and task target tags are two different dimensions of tags used for different purposes. During the data acquisition phase, multiple participants are typically recruited, for example, 20 to 30 subjects. The brain-computer interface system can record which specific subject each segment of physiological signal data originated from, thereby obtaining the user tags.

[0061] Secondly, user classification loss can be determined based on individual physiological characteristics and user tags. User classification loss is a loss function used to measure the accuracy of user identity prediction.

[0062] It is understandable that the greater the difference between individual physiological characteristics and user tags, the greater the user classification loss; the smaller the difference between individual physiological characteristics and user tags, the smaller the user classification loss.

[0063] Next, the target loss is determined based on the target classification loss, orthogonal constraint loss, and user classification loss. Here, the target loss can be determined based on the sum of the target classification loss, orthogonal constraint loss, and user classification loss, or it can be determined based on the weighted sum of the target classification loss, orthogonal constraint loss, and user classification loss. This embodiment of the invention does not specifically limit this method.

[0064] The method provided in this invention determines user classification loss based on individual physiological characteristics and user tags. It can explicitly guide the individual representation expert module to actively learn and aggregate individualized physiological patterns that can distinguish user identities. This makes the individual physiological feature vector richer in individual-specific information, and the orthogonal constraints applied subsequently can more effectively separate this information from the common features of the task, achieving more thorough feature decoupling and further improving the cross-user generalization ability and robustness of the finally trained target discrimination model.

[0065] Based on the above embodiments, the step 110 of acquiring multimodal physiological signal sample data includes: Step 1101: Obtain raw physiological signal data including multiple task target categories; Step 1102: Extract the real physiological signal data corresponding to the target category with sparse frequency in the original physiological signal data as seed data; Step 1103: Input the seed data into a preset conditional diffusion generation model to obtain the synthetic physiological signal data of the seed data generated by the conditional diffusion generation model; Step 1104: Combine the original physiological signal data with the synthesized physiological signal data to obtain the multimodal physiological signal sample data.

[0066] Specifically, in real continuous monitoring scenarios, the frequency of the target objects that users need to pay attention to is much lower than that of the non-target background. This inherent sparsity leads to a serious imbalance in the ratio of positive and negative samples in the training dataset. Traditional models often become biased as a result, making it difficult to learn robust decision boundaries.

[0067] To address the severe imbalance between positive and negative samples caused by the extremely low frequency of target appearance in continuous video streams, this embodiment introduces a progressive conditional diffusion generation model in the offline stage to implement high-fidelity joint generation of EEG and eye-tracking signals. Since image data can be augmented with high quality using conventional methods such as flipping and cropping, the data augmentation stage in this embodiment primarily targets EEG and eye-tracking data. Let the preprocessed seed data be the joint matrix. .in, This is the EEG feature matrix. This is the eye movement feature matrix (containing the coordinates of the fixation point and the pupil diameter). This represents the total number of channels. The length of the time series. This indicates the number of channels for acquiring EEG signals. This indicates the number of channels in the eye movement signal.

[0068] To address this problem, this embodiment proposes a data acquisition method based on generative data augmentation, specifically including: First, raw physiological signal data encompassing multiple task target categories can be acquired. This raw physiological signal data refers to real user data directly collected by hardware devices and preprocessed, without any generative enhancement. This embodiment aims to construct a dynamic visual search experimental paradigm for diverse continuous video stream applications such as UAV low-altitude reconnaissance, wide-area security monitoring, intelligent cockpit hazard warning, and visual retrieval of assistive devices for the disabled. The brain-computer interface system uses high-resolution (1080P) and high-frame-rate (60 fps) continuous video streams as visual stimuli, and performs spatiotemporal pseudo-random labeling of specific target objects, such as specific types of armored vehicles and abnormally loitering personnel, with non-target background interference.

[0069] In the multimodal data acquisition phase, the brain-computer interface (BCI) system establishes a microsecond-level global timestamp benchmark at its physical layer. Under this unified clock architecture, the BCI system uses a 64-lead high-fidelity EEG device to capture EEG signals from all brain regions of the subject in real time at a sampling rate of 1000 Hz. Simultaneously, a high-precision eye tracker is used at the same frequency of 1000 Hz to record the absolute coordinates of the subject's two-dimensional gaze points, pupil diameter, and dynamic saccade trajectories. At the same time, the BCI system's backend strictly records the absolute timestamp of each frame of video footage, thus ensuring precise temporal alignment of the "EEG-eye-video" trimodal signals during dynamic interaction at the data source.

[0070] For the acquired raw, multi-source, heterogeneous data, the brain-computer interface system executes a standardized and specific spatiotemporal joint preprocessing procedure: For EEG signal data, bilateral mastoid rereference and 0.1-40 Hz bandpass filtering are performed sequentially to remove power frequency and high-frequency background noise. Independent component analysis is applied to accurately separate and remove EEG and EMG artifacts. Subsequently, a temporal segment of [-200, 1000] ms is truncated with the target presentation time as the zero point, and baseline correction is performed. For eye movement signal data, after removing extreme values ​​caused by out-of-bounds movement and blinking, a velocity thresholding algorithm is applied to accurately extract fixation and saccade event features, and spatial motion compensation is performed on the eye movement coordinates in conjunction with the optical flow dynamics of the video background. The EEG and eye movement signal data sets obtained after the above processing constitute the raw physiological signal data.

[0071] After obtaining the raw physiological signal data, the real physiological signal data corresponding to the sparse target categories in the raw physiological signal data can be extracted as seed data.

[0072] Here, sparse target categories refer to target categories whose sample size is far less than other categories in the original physiological signal data. In the dynamic visual search experimental paradigm, the frequency of target occurrence is equivalently controlled within a fast sequence visual presentation rhythm of approximately 10 Hz, and the presentation probability is strictly limited to below 10%, thus constituting a typical low-probability Oddball target stimulus evoked paradigm. This experimental design itself results in sparse sample categories corresponding to the target.

[0073] Among them, real physiological signal data refers to physiological response data generated by users when observing target objects, which is filtered from raw physiological signal data.

[0074] After obtaining the seed data, it can be input into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model.

[0075] In this embodiment, the conditional diffusion generation model generates new data from pure noise by learning a denoising process. This model employs a progressive conditional diffusion model to achieve high-fidelity joint generation of EEG and eye-tracking signals.

[0076] Here, the synthetic physiological signal data is artificial data generated by a conditional diffusion generation model that is highly similar to, but not exactly the same as, the real seed data in terms of statistical distribution and characteristics.

[0077] Specifically, after the conditional diffusion generative model training converges, the brain-computer interface system initiates an offline inference process to generate massive amounts of high-fidelity synthetic physiological signal data. The specific generation and inference steps are as follows: (1) Conditional sampling and perturbation: The combination of macroscopic structural prior condition matrix and microscopic detail constraint tensor corresponding to the target category is randomly extracted from real physiological signal data. And apply controlled Gaussian perturbations to it to generate variant condition vectors. .

[0078] (2) Two-stage reverse generation reasoning: First, pure noise is generated. With the variant condition vector The input is fed into the trained first-level network, and an initial synthesized signal is generated through N steps of inverse denoising. Secondly, the preliminary synthesized signal With the variant condition vector and the corresponding task target category tags Together as refined constraints The data is input into the trained second-level network for inverse reconstruction, and finally outputs complete synthetic physiological signal data. .

[0079] Finally, the raw physiological signal data and the synthesized physiological signal data can be merged to obtain multimodal physiological signal sample data. The synthesized physiological signal data generated in the previous step is then decoupled dimensionally to form synthetic EEG samples. With synthetic eye-tracking samples and associate it with the corresponding task objective category label. After binding, the data is added to the original physiological signal dataset, thereby bridging the distribution gap between positive and negative samples at the physical feature level, and finally obtaining multimodal physiological signal sample data with a more balanced class distribution.

[0080] The method provided in this invention inputs seed data into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model. Before training begins, it can specifically perform high-quality data augmentation for target categories with insufficient sample size, thereby effectively alleviating the problem of extreme imbalance between positive and negative samples caused by the natural sparsity of the target and avoiding information loss caused by traditional downsampling methods.

[0081] Based on the above embodiments, step 1103 includes: The conditional diffusion generation model determines the macroscopic structure prior condition matrix based on the low-frequency amplitude envelope extracted from the seed data. The time-frequency energy distribution and instantaneous phase are extracted from the seed data as microscopic detail constraint condition tensors. Based on the macroscopic structure prior condition matrix, a first-stage inverse denoising process is performed on Gaussian noise to generate a preliminary synthesized signal with a global temporal trend. Based on the preliminary synthesized signal and the microscopic detail constraint condition tensor, a second-stage inverse denoising process is performed to progressively reconstruct and generate the synthesized physiological signal data.

[0082] Specifically, the conditional diffusion generation model determines the macroscopic structural prior condition matrix based on the low-frequency amplitude envelope extracted from the seed data, and extracts the time-frequency energy distribution and instantaneous phase from the seed data as microscopic detail constraint condition tensors.

[0083] Among them, the low-frequency amplitude envelope is used to reflect the macroscopic dynamic trend of physiological signals in seed data. (Targeting...) Analytical signals are constructed using the Hilbert Transform. : ; in, This represents the time-domain signal corresponding to seed data on a single physiological acquisition channel. The imaginary unit, For Hilbert transform operators, Represents the integral dummy variable. This represents time. The low-frequency amplitude envelope of the signal is then extracted from this. Low-frequency amplitude envelope characterizes the slow-wave trend of ERPs in EEG signals and the macroscopic saccade trajectory of eye movements.

[0084] Here, the macroscopic structure prior condition matrix is ​​a matrix composed of the low-frequency amplitude envelopes extracted from all channels in the seed data, denoted as... The macroscopic structure prior condition matrix provides the first stage of the conditional diffusion generation model with prior knowledge about the overall shape and trend of the signal.

[0085] Secondly, the time-frequency energy distribution and instantaneous phase are extracted from the seed data as microscopic detail constraint tensors.

[0086] Among them, time-frequency energy distribution and instantaneous phase are used to reflect the fine structure and rhythmic changes of physiological signals in seed data at different times and frequencies. Conditional diffusion generation models are used for... Performing a continuous wavelet transform (CWT) captures high-frequency rhythms of EEG and microsaccades of eye movements: ; In the formula, Represents the continuous wavelet transform coefficients. It is a complex number. For frequency scaling factor, The time shift factor, It is the conjugate of the complex Morlet mother wavelet. The time-frequency energy distribution is then calculated from this. With instantaneous phase ,in, Indicates the intensity of oscillation. express The imaginary part, express The real part.

[0087] Here, the microscopic detail constraint tensor is a tensor formed by splicing the time-frequency energy distribution and the instantaneous phase, denoted as The micro-detail constraint tensor provides precise constraints on the local details and high-frequency components of the signal for the second stage of the conditional diffusion generation model.

[0088] Furthermore, the conditional diffusion generation model, based on the macroscopic structure prior condition matrix, performs a first-stage inverse denoising process on Gaussian noise to generate a preliminary synthesized signal with a global temporal trend. This is the first stage of the "coarse-to-fine" generation process. Starting with purely random Gaussian noise, the conditional diffusion generation model, guided by the macroscopic structure prior condition matrix, progressively denoises, aiming to generate an intermediate result similar to the real signal in its low-frequency envelope. The preliminary synthesized signal is the output of the first stage. This preliminary synthesized signal already possesses the correct macroscopic morphology, such as the basic shape of the ERP slow wave, but lacks high-frequency physiological details, and can be denoted as... Then, based on the initial synthesized signal and the micro-detail constraint tensor, a second stage of inverse denoising is performed to progressively reconstruct and generate synthetic physiological signal data. This is the second stage, or "detail injection." The conditional diffusion generation model, based on the initial synthesized signal generated in the first stage, is further guided by the micro-detail constraint tensor to continue inverse denoising. The goal of this stage is to accurately inject the high-frequency rhythms and local details contained in the real physiological signal into the signal while maintaining the correct macroscopic structure, ultimately outputting high-fidelity synthetic physiological signal data.

[0089] The method provided in this invention extracts the low-frequency amplitude envelope as the prior condition matrix of the macroscopic structure and extracts the time-frequency energy distribution and instantaneous phase as the constraint tensor of the microscopic details. Based on this, a two-stage progressive reconstruction is performed, decomposing the complex task of signal generation into two easier-to-learn sub-tasks: macroscopic structure generation and microscopic detail injection. This ensures that the generated synthetic physiological signal data is not only consistent with the real data in statistical distribution, but also achieves high fidelity in key physical characteristics that determine signal quality, such as the slow wave trend of ERPs and high-frequency rhythm, significantly improving the quality of data augmentation.

[0090] Based on the above embodiments, the training steps of the conditional diffusion generation model include: Step 21, determine the initial conditional diffusion generation model; the initial conditional diffusion generation model includes an initial first-level network and an initial second-level network; Step 22: Obtain sample seed data, and gradually inject Gaussian noise into the sample seed data to obtain multi-step noisy data; Step 23: Input the macroscopic structure prior condition matrix corresponding to the sample seed data and the noisy data corresponding to the first stage time step in the multi-step noisy data into the initial first-level network to obtain the first noise prediction result and preliminary synthesized signal output by the initial first-level network; Step 24: Based on the difference between the first noise prediction result and the Gaussian noise, determine the first training loss, and perform parameter iteration on the initial first-level network based on the first training loss to obtain the trained first-level network; Step 25: Obtain the sample task target category label corresponding to the sample seed data; Step 26: Input the micro-detail constraint tensor corresponding to the sample seed data, the sample task target category label, the preliminary synthesized signal, and the noisy data corresponding to the second stage time step in the multi-step noisy data into the initial second-level network to obtain the second noise prediction result output by the initial second-level network; Step 27: Based on the difference between the second noise prediction result and the Gaussian noise, determine the second training loss, and perform parameter iteration on the initial second-level network based on the second training loss to obtain the trained second-level network; Step 28: Based on the trained first-level network and the trained second-level network, determine the conditional diffusion generation model.

[0091] Specifically, firstly, the initial conditional diffusion generation model is determined, which includes an initial first-level network and an initial second-level network.

[0092] Here, the initial conditional diffusion generative model refers to a model whose internal network parameters are randomly initialized or obtained through pre-training before training begins. The initial conditional diffusion generative model is a cascaded U-Net network structure that includes forward noise addition and two-stage inverse denoising. The initial first-level network and the initial second-level network in the initial conditional diffusion generative model perform the first-stage macroscopic structure generation and the second-stage microscopic detail injection, respectively.

[0093] Then, seed data can be obtained, and Gaussian noise is progressively injected into the seed data to obtain multi-step noisy data. The seed data is the real data used to train the conditional diffusion generation model. This step is the standard forward pass procedure for training the diffusion model. During the training phase, the seed data... A Markov chain is defined as a chain that gradually injects Gaussian noise over N time steps. ; In the formula, This represents the conditional probability distribution of the forward diffusion process, i.e., the probability distribution of the state from the previous time step during the forward diffusion process. Next time step The probability density function of the transition follows a mean of Covariance is The normal distribution; , This represents the total number of time steps in the forward diffusion process. This represents the noisy sample data at step n-1 in the forward diffusion process. This represents the current time step in the forward diffusion process. For variance scheduling parameters, This is the identity matrix. After this process, a clean sample seed data... It will become a series of multi-step noisy data, ranging from slight noise to pure noise. , , ..., .

[0094] Furthermore, the macroscopic structural prior condition matrix corresponding to the sample seed data, and the noisy data corresponding to the first time step in the multi-step noisy data, can be input into the initial first-level network to obtain the first noise prediction result and the preliminary synthesized signal output by the initial first-level network. This step corresponds to training the initial first-level network. The first noise prediction result is the noise prediction result output by the initial first-level network.

[0095] The initial goal of the first-level network is to learn how to predict added noise from noisy data. The initial first-level network is trained using a macroscopic structure prior condition matrix as the guiding variable, learning to recover the envelope base containing slow EEG waves and large-scale eye movement displacements from the noise. (Preliminary synthesized signal). Mathematically expressed as: ; in, This represents the conditional probability distribution of the first-stage reverse diffusion process. This represents the intermediate signal predicted by the initial first-level network at the previous time step (after one denoising step) during the first-stage reverse diffusion process. This represents the noisy data at step n in the first stage of the reverse diffusion process. This represents the prior condition matrix of the macroscopic structure. This represents the prediction function for the mean of the backward diffusion distribution. This represents all the learnable parameters of the initial first-level network.

[0096] Then, based on the difference between the first noise prediction result and the Gaussian noise, a first training loss is determined, and the parameters of the initial first-level network are iterated based on the first training loss to obtain the trained first-level network. Here, the first training loss is a function used to measure the difference between the first noise prediction result and the actual injected Gaussian noise, such as mean squared error. The parameters of the initial first-level network are updated based on the first training loss using optimization algorithms such as backpropagation and gradient descent. This process is repeated until the network converges, thus obtaining the trained first-level network.

[0097] Then, obtain the sample task target category label corresponding to the sample seed data. The sample task target category label (denoted as...) The target category is the one to which the seed data used for training belongs. The target category label of the sample will be used as additional conditional information during the training of the second-level network.

[0098] The micro-detail constraint tensor corresponding to the sample seed data, the sample task target category label, the preliminary synthesized signal, and the noisy data corresponding to the second stage time step in the multi-step noisy data are input into the initial second-level network to obtain the second noise prediction result output by the initial second-level network. This step corresponds to training the second-level network. The preliminary synthesized signal is then input into the network. Microscopic detail constraint tensor and target category labels splicing into joint constraints The initial second-level network is trained to compensate for high-frequency physiological rhythms on the macroscopic framework, mathematically expressed as: ; in, This represents the conditional probability distribution of the second-stage reverse diffusion process. This indicates the initial synthesized signal. Indicates joint constraints, This represents the intermediate signal predicted by the initial second-stage network at the previous time step (after further denoising) during the second-stage back diffusion process. This represents the noisy data at step n in the second-stage reverse diffusion process. This represents the covariance of the second-stage backward diffusion distribution. This represents the second-order backward diffusion distribution mean prediction function.

[0099] After obtaining the second noise prediction result, the second training loss can be determined based on the difference between the second noise prediction result and the Gaussian noise, and the parameters of the initial second-level network can be iterated based on the second training loss to obtain the trained second-level network.

[0100] It is understandable that the greater the difference between the second noise prediction result and the Gaussian noise, the greater the second training loss; the smaller the difference between the second noise prediction result and the Gaussian noise, the smaller the second training loss.

[0101] Finally, the conditional diffusion generative model can be determined based on the trained first-level network and the trained second-level network.

[0102] The method provided in this invention decomposes the training process of the conditional diffusion generative model into independent, phased training of the initial first-level network and the initial second-level network. This ensures that different parts of the conditional diffusion model can focus on learning signal features at different levels, such as macroscopic structure and microscopic details, avoiding the optimization difficulties caused by learning mixed targets in a single network. This guarantees that the finally trained conditional diffusion generative model has the ability to stably generate high-quality, high-fidelity synthetic data.

[0103] Considering that after training the target discrimination model, it can be applied to perform multi-type target discrimination.

[0104] Based on the above embodiments, the present invention provides a method for identifying multiple types of targets. Figure 2 This is a flowchart illustrating the multi-type target discrimination method provided by the present invention, as shown below. Figure 2 As shown, the method includes: Step 210: Obtain the multimodal physiological data to be identified; the multimodal physiological data to be identified includes video data of the target to be identified, EEG data corresponding to the target to be identified, and eye movement data corresponding to the target to be identified. Step 220: Input the multimodal physiological data to be discriminated into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; The target discrimination model is obtained by performing the training method based on the target discrimination model of the brain-computer interface described above.

[0105] Specifically, firstly, the multimodal physiological data to be discriminated can be acquired. This multimodal physiological data is a set of data reflecting the user's real-time physiological state information regarding a specific target in the current real-time monitoring scenario.

[0106] Here, the multimodal physiological data to be discriminated may include video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye-tracking data corresponding to the target to be discriminated. The video data of the target to be discriminated reflects its visual features; for example, a real-time video stream containing video frames of an object. The EEG data corresponding to the target to be discriminated reflects the electrical activity of the user's cerebral cortex when the user observes the target. The eye-tracking data corresponding to the target to be discriminated reflects information related to the user's visual attention and processing when observing the target, such as fixation point and pupillary changes; this embodiment of the invention does not specifically limit these aspects.

[0107] After obtaining the multimodal physiological data to be discriminated, the data can be input into the target discrimination model to obtain the target discrimination prediction results output by the target discrimination model.

[0108] Here, the target discrimination model is a model with strong cross-user generalization ability and robustness obtained after iterative training based on the training method of the target discrimination model of the brain-computer interface mentioned above.

[0109] The target discrimination prediction result reflects the prediction information of the target discrimination model regarding the category to which the target to be discriminated belongs. For example, the target discrimination prediction result can be a probability value, representing the probability that the target to be discriminated is the target that the user is interested in; or in a multi-class task, the target discrimination prediction result can be a probability distribution, representing the probability that the target to be discriminated belongs to each preset category.

[0110] The method provided in this invention acquires multimodal physiological data to be discriminated, wherein the multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye-tracking data corresponding to the target to be discriminated; the multimodal physiological data to be discriminated is input into a target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; the target discrimination model is obtained by training the target discrimination model based on the above-mentioned brain-computer interface, thereby achieving calibration-free, high-precision multi-type target discrimination for new users.

[0111] Based on any of the above embodiments, step 220 includes: Step 2201: Obtain the real-time signal quality of all modalities in the multimodal physiological data to be judged; Step 2202: The real-time signal quality is input into the target discrimination model. In response to determining that the real-time signal quality of the target modality among all modalities is lower than a preset robustness threshold, the target discrimination model identifies the target modality as a missing modality. Based on the signal of at least one intact modality other than the missing modality in the physiological data to be discriminated, the missing modality representation corresponding to the missing modality is reconstructed in the latent space of the parallel encoder in the target discrimination model. Based on the missing modality representation, a compensating multimodal feature is determined, and the target discrimination prediction result is output based on the compensating multimodal feature.

[0112] Specifically, firstly, the real-time signal quality of all modalities in the multimodal physiological data to be discriminated is acquired. The real-time signal quality reflects the quality information of each currently acquired modal signal data. Let the complete set of modalities in the multimodal physiological data to be discriminated input to the brain-computer interface system be denoted as . ,in, This represents video data representing the target to be identified. This represents the EEG data corresponding to the target to be identified. This represents the eye movement data corresponding to the target to be identified.

[0113] Secondly, the real-time signal quality is input into the target discrimination model. The model determines the target modality as a missing modality if its real-time signal quality falls below a preset robustness threshold. The brain-computer interface system calculates quality evaluation metrics for each modality signal in real time, such as signal-to-noise ratio or energy entropy, and defines modal state vectors. The decision logic is as follows: ; In the formula, For modality Real-time signal quality, The preset robustness threshold, Representing modes Signal availability status indicator.

[0114] Furthermore, the target discrimination model reconstructs the missing modality representation corresponding to the missing modality in the latent space of the parallel encoder in the target discrimination model based on the signal of at least one intact modality other than the missing modality in the multimodal physiological data to be discriminated. Based on the missing modality representation, the compensating multimodal features are determined, and the target discrimination prediction result is output based on the compensating multimodal features.

[0115] Here, the target mode is the specific mode used to reflect the quality of the signal currently being evaluated. The missing mode is used to reflect its real-time signal quality. Below the preset robustness threshold This refers to modes that are deemed unreliable or have failed.

[0116] When the target mode failure is detected, i.e. At that time, the latent space feature reconstruction procedure is initiated. Let... For the currently available intact modes (Right now The representation vector in the hidden space.

[0117] Here, the missing modality representation is used to reflect the vector information that can replace the true missing modality features, inferred and reconstructed in the latent space using feature information from one or more intact modalities. The reconstructed missing modality representation. Determined by the following linear transformation equation: ; In the formula, and The cross-modal transformation weight matrix obtained through pre-training. The dimension of the latent space representation vector is represented by the vector. For bias vectors, These are weighting coefficients based on modal correlation. Indicates the currently available intact modes The representation vector in the hidden space. This equation physically reconstructs the information entropy of the missing mode within the manifold space by utilizing the statistical correlation of existing modes.

[0118] Here, compensated multimodal features are used to reflect the complete feature vector information obtained after fusing the true features of the intact modalities with the reconstructed representations of the missing modalities, which is then used for subsequent decoding. After reconstruction, compensated multimodal features are constructed. The formula is as follows: ; in, , This represents the complete modality set of the multimodal physiological data to be discriminated. This indicates a feature cascade or weighted fusion operator. Representing modes Signal availability status indicator This indicates the missing modality representation. Representing modes The original hidden representation vector. This mechanism ensures that even under conditions of sensor detachment or extreme environmental occlusion, the subsequent decoding process can still obtain input features with consistent dimension and semantic alignment.

[0119] The method provided in this invention reconstructs the missing modal representation corresponding to the missing modality in the latent space of the parallel encoder in the target discrimination model, and determines the compensating multimodal features based on the missing modal representation, so that the discrimination method has the ability to cope with the interruption of single or partial modal signals. Thus, without interrupting the discrimination task, the remaining intact modal information can be used to compensate for the missing modal information, which greatly enhances the robustness of multi-type target discrimination methods in real complex environments.

[0120] Based on the above embodiments, the target discrimination model further includes a dynamic gating network, and a common expert module, an individual representation expert module, and a modal complementarity expert module connected to the dynamic gating network; Step 220 also includes: Step 220-1: Input the multimodal physiological data to be discriminated into the latent space of the parallel encoder for encoding to obtain the latent space representation vector; Step 220-2: Input the latent space representation vector into the dynamic gating network to obtain the first routing weight corresponding to the common expert module, the second routing weight corresponding to the individual representation expert module, and the third routing weight corresponding to the modal complementarity expert module output by the dynamic gating network. Step 220-3: Based on the first routing weight, input the latent space representation vector into the commonality expert module to obtain the cross-user task commonality prediction features output by the commonality expert module; Step 220-4: Based on the second routing weight, input the latent space representation vector into the individual representation expert module to obtain the individual physiological prediction features of the current user output by the individual representation expert module; Step 220-5: Based on the third routing weight, input the latent space representation vector to the modality complementarity expert module to obtain the interaction information between multimodal signals in the multimodal physiological data to be discriminated, output by the modality complementarity expert module; Step 220-6: Based on the common prediction features of the task, the individual physiological prediction features, and the interaction information, determine the target discrimination prediction result.

[0121] Specifically, the target discrimination model also includes a dynamic gating network, as well as a common expert module, an individual representation expert module, and a modal complementarity expert module connected to the dynamic gating network.

[0122] Accordingly, the multimodal physiological data to be discriminated can be input into the latent space of the parallel encoder for encoding to obtain the latent space representation vector.

[0123] The latent space representation vector refers to the corresponding feature vector output by the parallel encoder after mapping the original physiological signals of each modality to a shared low-dimensional real-valued vector space.

[0124] After obtaining the latent space representation vector, the latent space representation vector can be input into the dynamic gating network to obtain the first routing weight corresponding to the common expert module, the second routing weight corresponding to the individual representation expert module, and the third routing weight corresponding to the modal complementarity expert module output by the dynamic gating network.

[0125] Here, the first routing weight reflects the contribution information of the latent space representation vector to the common expert module for processing. The second routing weight reflects the contribution information of the latent space representation vector to the individual representation expert module for processing. The third routing weight reflects the contribution information of the latent space representation vector to the modal complementarity expert module for processing.

[0126] Furthermore, based on the first routing weight, the latent space representation vector is input into the commonality expert module to obtain the cross-user task commonality prediction features output by the commonality expert module. These task commonality prediction features are used to reflect core task-induced information independent of individual differences, aiming to capture core neural response patterns independent of individual differences.

[0127] Based on the second routing weight, the latent space representation vector is input to the individual representation expert module to obtain the individual physiological prediction features of the current user output by the individual representation expert module. These individual physiological prediction features are used to capture the physiological bias and non-stationary noise floor of the current specific user.

[0128] Based on the third routing weight, the latent space representation vector is input to the modality complementarity expert module to obtain the interaction information between multimodal signals in the multimodal physiological data to be discriminated, output by the modality complementarity expert module. This interaction information reflects the correlation features that reflect the mutual enhancement or complementarity between different modal signals such as EEG and eye movement.

[0129] Finally, the target discrimination prediction result can be determined based on common prediction features of the task, individual physiological prediction features, and interaction information. This step integrates the features output by all expert modules through a final classifier or fusion layer to generate the final target discrimination prediction result.

[0130] Furthermore, to address the data distribution drift issue that may arise from changes in user physiological state or environment, this embodiment also provides an online adaptive fine-tuning strategy: For the multimodal physiological data to be discriminated Brain-computer interface systems can use target discrimination models to predict probability distributions. Minimizing information entropy as a self-supervised optimization objective: ; in, This represents the loss from self-supervised optimization. This indicates that the target discrimination model predicts the category to which the multimodal physiological data to be discriminated belongs. The probability of.

[0131] The self-supervised optimization loss guides the target discrimination model to quickly fit the boundary distribution of the current data stream.

[0132] During online adaptation, the brain-computer interface system performs strict parameter control, completely freezing the parameters of the common expert module and parallel encoder to preserve general target classification capabilities; only backpropagation is allowed to update the dynamic gating network and individual representation expert modules. This linked fine-tuning mechanism ensures that the brain-computer interface system can achieve real-time dynamic adaptation to new users and incomplete modal inputs without recalibration by updating individual bias terms.

[0133] The training apparatus for the target discrimination model of the brain-computer interface provided by the present invention will be described below. The training apparatus for the target discrimination model of the brain-computer interface described below and the training method for the target discrimination model of the brain-computer interface described above can be referred to in correspondence.

[0134] Based on any of the above embodiments, the present invention provides a training device for a target discrimination model of a brain-computer interface. Figure 3 This is a schematic diagram of the structure of the training device for the target discrimination model of the brain-computer interface provided by the present invention, as shown below. Figure 3 As shown, the device includes: The first acquisition unit 310 is used to acquire multimodal physiological signal sample data and task target labels corresponding to the multimodal physiological signal sample data; The first input unit 320 is used to input the multimodal physiological signal sample data into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model. The first determining unit 330 is used to determine the target classification loss based on the common features of the task and the target label of the task; The second determining unit 340 is used to determine the orthogonal constraint loss based on the orthogonality of the common features of the task and the individual physiological features in the feature space; The update unit 350 is used to determine the target loss based on the target classification loss and the orthogonal constraint loss, and update the model parameters of the initial decoding model based on the target loss.

[0135] The apparatus provided in this embodiment of the invention acquires multimodal physiological signal sample data and corresponding task target labels; inputs the multimodal physiological signal sample data into an initial decoding model including a common feature expert module and an individual representation expert module to output task common features and individual physiological features respectively; determines the target classification loss based on the task common features and task target labels; determines the orthogonal constraint loss based on the orthogonality of the task common features and individual physiological features in the feature space; determines the target loss based on the target classification loss and the orthogonal constraint loss together, and updates the model parameters of the initial decoding model. This invention introduces orthogonal constraint loss during model training to forcibly separate common task features from individual physiological features. This effectively decouples cross-user common features directly related to task target discrimination from individual physiological features reflecting user-specific physiological backgrounds during the model training phase. Consequently, the target discrimination model can learn higher-purity task discrimination features independent of individual differences, improving the generalization ability and robustness of the target discrimination model when faced with data distribution shifts in unknown new subjects. This solves the problem that the model cannot be applied to multi-class target detection tasks in continuous dynamic video streams due to stimulus paradigm limitations and interference from individual differences, ensuring the decoding performance of the brain-computer interface system in cross-user application scenarios.

[0136] Based on any of the above embodiments, the updating unit 350 is specifically used for: Obtain the user tags corresponding to the multimodal physiological signal sample data; Based on the individual physiological characteristics and the user tags, determine the user classification loss; The target loss is determined based on the target classification loss, the orthogonal constraint loss, and the user classification loss.

[0137] Based on any of the above embodiments, the first acquisition unit 310 specifically includes: A data acquisition unit is used to acquire raw physiological signal data including multiple task target categories; The seed data extraction unit is used to extract the real physiological signal data corresponding to the target category with sparse frequency in the original physiological signal data as seed data; A generation unit is used to input the seed data into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model. The merging unit is used to merge the original physiological signal data with the synthesized physiological signal data to obtain the multimodal physiological signal sample data.

[0138] Based on any of the above embodiments, the generation unit is specifically used for: The conditional diffusion generation model determines the macroscopic structure prior condition matrix based on the low-frequency amplitude envelope extracted from the seed data. The time-frequency energy distribution and instantaneous phase are extracted from the seed data as microscopic detail constraint condition tensors. Based on the macroscopic structure prior condition matrix, a first-stage inverse denoising process is performed on Gaussian noise to generate a preliminary synthesized signal with a global temporal trend. Based on the preliminary synthesized signal and the microscopic detail constraint condition tensor, a second-stage inverse denoising process is performed to progressively reconstruct and generate the synthesized physiological signal data.

[0139] Based on any of the above embodiments, a training unit is further included, wherein the training unit is specifically used for: Determine the initial conditional diffusion generation model; the initial conditional diffusion generation model includes an initial first-level network and an initial second-level network; Obtain sample seed data, and gradually inject Gaussian noise into the sample seed data to obtain multi-step noisy data; The macroscopic structure prior condition matrix corresponding to the sample seed data and the noisy data corresponding to the first stage time step in the multi-step noisy data are input into the initial first-level network to obtain the first noise prediction result and the preliminary synthesized signal output by the initial first-level network. Based on the difference between the first noise prediction result and the Gaussian noise, a first training loss is determined, and the parameters of the initial first-level network are iterated based on the first training loss to obtain a trained first-level network. Obtain the sample task target category label corresponding to the sample seed data; The micro-detail constraint tensor corresponding to the sample seed data, the sample task target category label, the preliminary synthesized signal, and the noisy data corresponding to the second stage time step in the multi-step noisy data are input into the initial second-level network to obtain the second noise prediction result output by the initial second-level network. Based on the difference between the second noise prediction result and the Gaussian noise, a second training loss is determined, and the parameters of the initial second-level network are iterated based on the second training loss to obtain a trained second-level network. Based on the trained first-level network and the trained second-level network, the conditional diffusion generation model is determined.

[0140] The multi-type target discrimination device provided by the present invention is described below. The multi-type target discrimination device described below can be referred to in correspondence with the multi-type target discrimination method described above.

[0141] Based on any of the above embodiments, the present invention provides a multi-type target discrimination device. Figure 4 This is a schematic diagram of the structure of the multi-type target discrimination device provided by the present invention, as shown below. Figure 4 As shown, the device includes: The second acquisition unit 410 is used to acquire multimodal physiological data to be identified; the multimodal physiological data to be identified includes video data of the target to be identified, EEG data corresponding to the target to be identified, and eye movement data corresponding to the target to be identified. The second input unit 420 is used as an input unit to input the multimodal physiological data to be discriminated into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; The target discrimination model is obtained by performing the training method based on the target discrimination model of the brain-computer interface described above.

[0142] The apparatus provided in this invention acquires multimodal physiological data to be discriminated, wherein the multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye movement data corresponding to the target to be discriminated; the multimodal physiological data to be discriminated is input into a target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; the target discrimination model is obtained by training the target discrimination model based on the above-mentioned brain-computer interface, thereby achieving calibration-free, high-precision multi-type target discrimination for new users.

[0143] Based on any of the above embodiments, the second input unit 420 is specifically used for: Obtain the real-time signal quality of all modalities in the multimodal physiological data to be discriminated; The real-time signal quality is input into the target discrimination model. In response to determining that the real-time signal quality of the target modality among all modalities is lower than a preset robustness threshold, the target modality is identified as a missing modality. Based on the signal of at least one intact modality other than the missing modality in the physiological data of the multimodalities to be discriminated, the missing modality representation corresponding to the missing modality is reconstructed in the latent space of the parallel encoder in the target discrimination model. Based on the missing modality representation, a compensating multimodal feature is determined, and the target discrimination prediction result is output based on the compensating multimodal feature.

[0144] Based on any of the above embodiments, the target discrimination model further includes a dynamic gating network, and a common expert module, an individual representation expert module, and a modal complementarity expert module connected to the dynamic gating network; It also includes a third input unit, which is specifically used for: The multimodal physiological data to be discriminated is input into the latent space of the parallel encoder for encoding to obtain the latent space representation vector; The latent space representation vector is input into the dynamic gating network to obtain the first routing weight corresponding to the common expert module, the second routing weight corresponding to the individual representation expert module, and the third routing weight corresponding to the modal complementarity expert module output by the dynamic gating network. Based on the first routing weight, the latent space representation vector is input to the commonality expert module to obtain the cross-user task commonality prediction features output by the commonality expert module; Based on the second routing weight, the latent space representation vector is input to the individual representation expert module to obtain the individual physiological prediction features of the current user output by the individual representation expert module; Based on the third routing weight, the latent space representation vector is input to the modal complementarity expert module to obtain the interaction information between multimodal signals in the multimodal physiological data to be discriminated, which is output by the modal complementarity expert module. Based on the common prediction features of the task, the individual physiological prediction features, and the interaction information, the target discrimination prediction result is determined.

[0145] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logic instructions in the memory 530 to execute a training method for a target discrimination model of the brain-computer interface. This method includes: acquiring multimodal physiological signal sample data and task target labels corresponding to the multimodal physiological signal sample data; inputting the multimodal physiological signal sample data into an initial decoding model to obtain common task features output by a commonality expert module in the initial decoding model, and individual physiological features output by an individual representation expert module in the initial decoding model; determining a target classification loss based on the common task features and the task target labels; determining an orthogonal constraint loss based on the orthogonality of the common task features and the individual physiological features in the feature space; determining a target loss based on the target classification loss and the orthogonal constraint loss; and updating the model parameters of the initial decoding model based on the target loss.

[0146] The processor 510 can also call logical instructions in the memory 530 to execute a multi-type target discrimination method, which includes: acquiring multimodal physiological data to be discriminated; the multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye movement data corresponding to the target to be discriminated; inputting the multimodal physiological data to be discriminated into a target discrimination model to obtain a target discrimination prediction result output by the target discrimination model; the target discrimination model is obtained by training the target discrimination model based on the above-mentioned brain-computer interface.

[0147] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the training method of the target discrimination model of the brain-computer interface provided by the above methods. The method includes: acquiring multimodal physiological signal sample data and task target labels corresponding to the multimodal physiological signal sample data; inputting the multimodal physiological signal sample data into an initial decoding model to obtain the task common features output by the common expert module in the initial decoding model and the individual physiological features output by the individual representation expert module in the initial decoding model; determining a target classification loss based on the task common features and the task target labels; determining an orthogonal constraint loss based on the orthogonality of the task common features and the individual physiological features in the feature space; determining a target loss based on the target classification loss and the orthogonal constraint loss, and updating the model parameters of the initial decoding model based on the target loss.

[0149] When the computer program is executed by the processor, the computer can execute the multi-type target discrimination method provided by the above methods. The method includes: acquiring multimodal physiological data to be discriminated; the multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye movement data corresponding to the target to be discriminated; inputting the multimodal physiological data to be discriminated into a target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; the target discrimination model is obtained by training the target discrimination model based on the above brain-computer interface.

[0150] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a training method for a target discrimination model of a brain-computer interface provided by the methods described above. This method includes: acquiring multimodal physiological signal sample data and task target labels corresponding to the multimodal physiological signal sample data; inputting the multimodal physiological signal sample data into an initial decoding model to obtain common task features output by a commonality expert module in the initial decoding model, and individual physiological features output by an individual representation expert module in the initial decoding model; determining a target classification loss based on the common task features and the task target labels; determining an orthogonal constraint loss based on the orthogonality of the common task features and the individual physiological features in the feature space; determining a target loss based on the target classification loss and the orthogonal constraint loss; and updating the model parameters of the initial decoding model based on the target loss.

[0151] When executed by a processor, the computer program implements the multi-type target discrimination method provided by the above methods. The method includes: acquiring multimodal physiological data to be discriminated; the multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye movement data corresponding to the target to be discriminated; inputting the multimodal physiological data to be discriminated into a target discrimination model to obtain a target discrimination prediction result output by the target discrimination model; the target discrimination model is obtained by training the target discrimination model based on the above brain-computer interface.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A training method for a target discrimination model of a brain-computer interface, characterized in that, The target discrimination model is obtained by iteratively executing the following steps until a preset iteration termination condition is met: Acquire multimodal physiological signal sample data and corresponding task target labels; the multimodal physiological signal sample data is one or more sets of data reflecting the user's physiological state when performing a specific cognitive task; the multimodal physiological signal sample data includes electroencephalogram (EEG) signal data and eye movement (EMG) signal data; the task target label is used to identify whether each of the multimodal physiological signal sample data corresponds to the user's physiological response to a target stimulus; The multimodal physiological signal sample data is input into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model. Based on the common features of the tasks and the task target labels, the target classification loss is determined; Based on the orthogonality of the common features of the tasks and the individual physiological features in the feature space, the orthogonal constraint loss is determined. Based on the target classification loss and the orthogonal constraint loss, the target loss is determined, and the model parameters of the initial decoding model are updated based on the target loss; The step of determining the target loss based on the target classification loss and the orthogonal constraint loss includes: Obtain the user tag corresponding to the multimodal physiological signal sample data; the user tag is the identity information used to uniquely identify the user who provided the current multimodal physiological signal sample data; Based on the individual physiological characteristics and the user tags, determine the user classification loss; The target loss is determined based on the target classification loss, the orthogonal constraint loss, and the user classification loss.

2. The training method for the target discrimination model of the brain-computer interface according to claim 1, characterized in that, The acquisition of multimodal physiological signal sample data includes: Acquire raw physiological signal data including multiple task target categories; Extract the real physiological signal data corresponding to the target category with sparse frequency in the original physiological signal data as seed data; The seed data is input into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model. The original physiological signal data is combined with the synthesized physiological signal data to obtain the multimodal physiological signal sample data.

3. The training method for the target discrimination model of the brain-computer interface according to claim 2, characterized in that, The step of inputting the seed data into a preset conditional diffusion generation model to obtain synthetic physiological signal data of the seed data generated by the conditional diffusion generation model includes: The conditional diffusion generation model determines the macroscopic structure prior condition matrix based on the low-frequency amplitude envelope extracted from the seed data. The time-frequency energy distribution and instantaneous phase are extracted from the seed data as microscopic detail constraint condition tensors. Based on the macroscopic structure prior condition matrix, a first-stage inverse denoising process is performed on Gaussian noise to generate a preliminary synthesized signal with a global temporal trend. Based on the preliminary synthesized signal and the microscopic detail constraint condition tensor, a second-stage inverse denoising process is performed to progressively reconstruct and generate the synthesized physiological signal data.

4. The training method for the target discrimination model of the brain-computer interface according to claim 3, characterized in that, The training steps of the conditional diffusion generation model include: Determine the initial conditional diffusion generation model; the initial conditional diffusion generation model includes an initial first-level network and an initial second-level network; Obtain sample seed data, and gradually inject Gaussian noise into the sample seed data to obtain multi-step noisy data; The macroscopic structure prior condition matrix corresponding to the sample seed data and the noisy data corresponding to the first stage time step in the multi-step noisy data are input into the initial first-level network to obtain the first noise prediction result and the preliminary synthesized signal output by the initial first-level network. Based on the difference between the first noise prediction result and the Gaussian noise, a first training loss is determined, and the parameters of the initial first-level network are iterated based on the first training loss to obtain a trained first-level network. Obtain the sample task target category label corresponding to the sample seed data; The micro-detail constraint tensor corresponding to the sample seed data, the sample task target category label, the preliminary synthesized signal, and the noisy data corresponding to the second stage time step in the multi-step noisy data are input into the initial second-level network to obtain the second noise prediction result output by the initial second-level network. Based on the difference between the second noise prediction result and the Gaussian noise, a second training loss is determined, and the parameters of the initial second-level network are iterated based on the second training loss to obtain a trained second-level network. Based on the trained first-level network and the trained second-level network, the conditional diffusion generation model is determined.

5. A method for identifying multiple target types, characterized in that, include: Acquire the multimodal physiological data to be discriminated; The multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye movement data corresponding to the target to be discriminated; The multimodal physiological data to be discriminated is input into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model; The target discrimination model is obtained by training the target discrimination model of the brain-computer interface as described in any one of claims 1 to 4.

6. The multi-type target discrimination method according to claim 5, characterized in that, The step of inputting the multimodal physiological data to be discriminated into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model includes: Obtain the real-time signal quality of all modalities in the multimodal physiological data to be discriminated; The real-time signal quality is input into the target discrimination model. In response to determining that the real-time signal quality of the target modality among all modalities is lower than a preset robustness threshold, the target modality is identified as a missing modality. Based on the signal of at least one intact modality other than the missing modality in the physiological data of the multimodalities to be discriminated, the missing modality representation corresponding to the missing modality is reconstructed in the latent space of the parallel encoder in the target discrimination model. Based on the missing modality representation, a compensating multimodal feature is determined, and the target discrimination prediction result is output based on the compensating multimodal feature.

7. The multi-type target discrimination method according to claim 6, characterized in that, The target discrimination model also includes a dynamic gating network, and a common expert module, an individual representation expert module, and a modal complementarity expert module connected to the dynamic gating network; The step of inputting the multimodal physiological data to be discriminated into the target discrimination model to obtain the target discrimination prediction result output by the target discrimination model further includes: The multimodal physiological data to be discriminated is input into the latent space of the parallel encoder for encoding to obtain the latent space representation vector; The latent space representation vector is input into the dynamic gating network to obtain the first routing weight corresponding to the common expert module, the second routing weight corresponding to the individual representation expert module, and the third routing weight corresponding to the modal complementarity expert module output by the dynamic gating network. Based on the first routing weight, the latent space representation vector is input to the commonality expert module to obtain the cross-user task commonality prediction features output by the commonality expert module; Based on the second routing weight, the latent space representation vector is input to the individual representation expert module to obtain the individual physiological prediction features of the current user output by the individual representation expert module; Based on the third routing weight, the latent space representation vector is input to the modal complementarity expert module to obtain the interaction information between multimodal signals in the multimodal physiological data to be discriminated, which is output by the modal complementarity expert module. Based on the common prediction features of the task, the individual physiological prediction features, and the interaction information, the target discrimination prediction result is determined.

8. A training device for a target discrimination model of a brain-computer interface, characterized in that, include: The first acquisition unit is used to acquire multimodal physiological signal sample data and task target labels corresponding to the multimodal physiological signal sample data; The multimodal physiological signal sample data is one or more sets of data used to reflect the physiological state of a user when performing a specific cognitive task; the multimodal physiological signal sample data includes electroencephalogram (EEG) signal data and eye movement signal data; the task target label is used to identify whether each of the multimodal physiological signal sample data corresponds to the user's physiological response to the target stimulus; The first input unit is used to input the multimodal physiological signal sample data into the initial decoding model to obtain the common features of the task output by the common expert module in the initial decoding model, and the individual physiological features output by the individual representation expert module in the initial decoding model. The first determining unit is used to determine the target classification loss based on the common features of the task and the target label of the task; The second determining unit is used to determine the orthogonal constraint loss based on the orthogonality of the common features of the task and the individual physiological features in the feature space; An update unit is used to determine the target loss based on the target classification loss and the orthogonal constraint loss, and update the model parameters of the initial decoding model based on the target loss; The update unit is specifically used for: Obtain the user tag corresponding to the multimodal physiological signal sample data; the user tag is the identity information used to uniquely identify the user who provided the current multimodal physiological signal sample data; Based on the individual physiological characteristics and the user tags, determine the user classification loss; The target loss is determined based on the target classification loss, the orthogonal constraint loss, and the user classification loss.

9. A multi-type target discrimination device, characterized in that, include: The second acquisition unit is used to acquire the multimodal physiological data to be discriminated; The multimodal physiological data to be discriminated includes video data of the target to be discriminated, EEG data corresponding to the target to be discriminated, and eye movement data corresponding to the target to be discriminated; The second input unit is used as an input unit to input the multimodal physiological data to be discriminated into the target discrimination model and obtain the target discrimination prediction result output by the target discrimination model. The target discrimination model is obtained by training the target discrimination model of the brain-computer interface as described in any one of claims 1 to 4.

Images