184 results about "Electroencephalogram feature" patented technology

The invention relates to the technical field of electroencephalogram signal processing, in particular to an ontology brain wave audio auditory perception synchronous feedback method, device and system and electronic equipment. The method comprises the following steps: receiving electroencephalogram signals of a collected user on line from electroencephalogram collection equipment through an upper computer, obtaining electroencephalogram signals of a specified frequency band from the electroencephalogram signals, and extracting corresponding electroencephalogram characteristics; according to the electroencephalogram features or preset rhythm parameters, the electroencephalogram signals of the specified frequency band are segmented into a plurality of electroencephalogram segments, a plurality of audio expressions corresponding to the electroencephalogram segments are generated, and feature parameters of the audio expressions are determined according to the electroencephalogram features of the corresponding electroencephalogram segments; generating brain wave audio representation data according to the audio representation corresponding to the electroencephalogram signals of the one or more designated frequency bands, and obtaining brain wave audio according to the brain wave audio representation data. Therefore, the physiological suitability of nerve regulation and control and the artistic expressivity of audio generation are met at the same time, and organic unification of nerve regulation and control and audio generation is achieved.

The invention relates to the technical field of cognitive impairment detection, and discloses a cognitive impairment early warning method and device based on an electroencephalogram micro state and an eye movement trajectory, and the method comprises the following steps: S1, data acquisition, S2, electroencephalogram preprocessing, S3, electroencephalogram feature extraction, S4, eye movement feature extraction, S5, feature fusion, and S6, early warning judgment. According to the method, through independent convolution branch of electroencephalogram and eye movement features, a receptive field is expanded by utilizing cavity convolution to capture multi-scale features, and long-distance dependence is modeled by a self-attention layer; after tensor splicing, time sequence information is dynamically fused through a gating cycle unit (GRU), and cross-modal time correlation is captured. The method has the advantages that the multi-modal feature hierarchical extraction and self-adaptive modeling capability is enhanced, the complementarity fusion efficiency is optimized, meanwhile, by means of cavity convolution sparse connection, self-attention parameter sharing and GRU lightweight design, the model complexity and the calculation efficiency are balanced, and efficient feature representation is provided for cognitive impairment early warning.

The invention provides a transcranial stimulation control method, a transcranial stimulation control system and a computer readable storage medium, and the method comprises the steps: collecting an electroencephalogram parameter signal, and converting the electroencephalogram parameter signal into an electroencephalogram feature vector; inputting the electroencephalogram feature vector into a large language model, obtaining an initial electrical stimulation signal corresponding to the electroencephalogram feature vector from a stimulation electroencephalogram knowledge model according to the electroencephalogram feature vector, and generating an initial electrical stimulation signal scheme; the initial electrical stimulation signal is optimized through an attention mechanism according to historical response data of a user, a final electrical stimulation signal scheme is output, and the historical response data of the user comprises a historical electrical stimulation signal scheme and user feedback information; and outputting the final electrical stimulation signal scheme to a transcranial stimulation module. The method can improve the insomnia treatment efficiency of the user.

The invention discloses a voice decoding method, system and device based on electroencephalogram signals and a medium, and relates to the technical field of electroencephalogram signal processing.The method comprises the steps that reading electroencephalogram signals and voice signals in the reading process of a to-be-tested person and imagination electroencephalogram signals in the imagination reading process of the to-be-tested person are collected; inputting the reading electroencephalogram signals and the voice signals into the DRCL to generate electroencephalogram characteristics containing voice information; training the DBM by using the electroencephalogram characteristics as input and using the voice signals as output, and adjusting the DBM by using imaginary electroencephalogram signals to construct a voice synthesizer; a mapping relation between the imaginary electroencephalogram signals and the voice signals is generated through the voice synthesizer, and decoding from the electroencephalogram signals to the voice signals is completed; according to the method, a deep representation correlation learning method is provided, potential correlation between electroencephalogram and voice signals can be deeply mined through a multi-layer network structure, a complex mode which is difficult to recognize by a traditional model is captured, and the voice decoding process is more accurate.

The invention discloses an electroencephalogram signal automatic feature extraction method based on a convolutional neural network, which comprises the following steps of: acquiring original electroencephalogram signal data, and performing adaptive artifact suppression and channel amplitude dynamic re-calibration based on a channel spectrum curvature change rate; executing frequency band deformation reconstruction, determining a frequency band boundary according to the frequency spectrum curvature change rate sequence, and constructing a frequency deformation mapping function to generate a plurality of frequency band component tensors; inputting the frequency band component tensor into an improved ConvNeXt model, and extracting a multi-dimensional electroencephalogram feature tensor; cross-frequency-domain attention fusion is executed, and a fused electroencephalogram feature vector is constructed; performing task reasoning, and outputting an electroencephalogram task response result; and performing feature feedback reconstruction according to the frequency deformation consistency index and the prediction offset degree, and jointly updating deformation convolution kernel parameters. According to the method, the accuracy and stability of electroencephalogram feature extraction are improved, and the method is suitable for various electroencephalogram intelligent processing scenes such as cognitive state recognition and medical auxiliary analysis.

According to the multi-mode neural learning network model, method, device and medium, after electroencephalogram signals and original images are coded, electroencephalogram features and image features are obtained, then the similarity of the electroencephalogram features and the image features is calculated, comparative learning of the electroencephalogram signals and the original image signals is completed, and the accuracy of the electroencephalogram signals and the original image signals is improved. The encoding of the original image comprises the steps of obtaining a first primary feature of an embedded representation level based on the original image, obtaining a second primary feature of the embedded representation level based on the filtered image, and fusing and reasoning the primary features, so that the embedded representation level primary features of the original image and the filtered image are respectively extracted through a double-branch image path; and an attention gating mechanism is adopted to carry out embedded representation level fusion, so that self-adaptive adjustment of an image modal structure is realized, the problem of a single path of traditional image coding is solved, and the alignment capability of a visual modal is effectively enhanced.

The invention provides an electroencephalogram-myoelectricity collaborative limb movement intention decoding method and system based on a symmetric cross-modal attention network, and belongs to the technical field of neural rehabilitation engineering and movement intention decoding. Comprising the following steps: acquiring an electroencephalogram signal and an electromyographic signal to be decoded; inputting the electroencephalogram signals into an electroencephalogram channel specificity feature extraction network, and extracting multi-scale space-time oscillation features; the electromyographic signals are input into an electromyographic channel specificity feature extraction network, and dynamic time sequence mode features are extracted; inputting the electroencephalogram features and the myoelectricity features into a symmetric cross-modal attention module, carrying out bidirectional feature interaction and calibration, and generating fusion enhancement features; the symmetric cross-modal attention module realizes mutual enhancement and alignment between the electroencephalogram signals and the electromyographic signals by taking own features as queries and taking another modal feature as a key and a value through a cross attention mechanism; and inputting the fusion enhancement feature into a classifier, and decoding to obtain a corresponding motion intention category.

The invention discloses a method and a related device for closed-loop optimization of a lower limb motor imagery experiment normal form through dynamic mind training, and relates to the field of rehabilitation medicines.The method comprises the steps that electroencephalogram signals of a testee when the testee executes a mind fusion lower limb motor imagery task are collected and preprocessed, and preprocessed electroencephalogram signal data are obtained; based on a dynamic feature extraction technology of a sliding window, calculating electrophysiological indexes in real time; performing weighted fusion on the motor imagery definition, the concentration degree and the correctness degree through predefined weights to generate a comprehensive state score; on the basis of the comprehensive state score, dynamically adjusting a normal feeling and task stimulation parameter applied to the testee; based on the collected electroencephalogram signals, extracting optimal perception motion rhythm and cross-frequency coupling characteristics; and inputting into the trained two-dimensional time convolution network decoding model to obtain the lower limb motor imagery intention of the testee. According to the method, the evoked rate and the stability of the electroencephalogram characteristics of the stroke patient in the lower limb motor imagery task can be improved.

The invention discloses a device for awareness level evaluation and a storage medium. The apparatus comprises: a processor; the device realizes the following operations: collecting clinical information of a person to be assessed and a task state electroencephalogram signal under a target stimulation normal form; extracting frequency domain characteristics of a specific frequency band and spatial-temporal characteristics of a target event related potential based on the task state electroencephalogram signal, and combining the frequency domain characteristics and the spatial-temporal characteristics into a corresponding electroencephalogram topographic map; inputting the corresponding electroencephalogram topographic map into a multi-modal large language model, and performing image feature extraction by using an image encoder to obtain electroencephalogram features; inputting clinical information into the multi-modal large language model, and performing text feature extraction by using a text encoder to obtain text features; and performing cross-modal attention calculation fusion on the electroencephalogram features and the text features by using a cross-modal fusion module to realize consciousness evaluation so as to output a consciousness evaluation result. By means of the scheme, the consciousness level of the patient can be automatically and accurately evaluated.

The invention relates to the field of education, and discloses a teaching window control method which comprises the following steps: synchronously acquiring eye movement tracking data, electroencephalogram signals and teaching semantic flow intensity in real time through a multi-modal sensor to form a multi-modal perception data input source; inputting the multi-modal perception data into a quantum annealing optimization module, constructing a Hamiltonian model based on semantic coupling strength and physical display constraints, and generating a window layout candidate parameter set in parallel; and performing symplectic geometry manifold correction on the candidate parameter set, calculating a physical adaptability boundary of the compensation display device through a differential geometry contact coefficient, and screening out a feasible solution space meeting curvature constraint. According to the method, the eye movement track and the electroencephalogram characteristics are fused through the multi-mode sensing unit, real-time layout optimization driven by the cognitive state is achieved, the limitation of separation of physiological signals and interface design is broken through, and window arrangement is made to accurately accord with the instantaneous cognitive load level of a learner.

The invention discloses an electroencephalogram voice annotation calibration and scheduling system and method for individual differences. The system comprises an electroencephalogram signal collecting and preprocessing module, a voice instruction receiving and recognizing module, a rapid individual calibration module, a dynamic task scheduling module, an electroencephalogram feature online decoding and annotation module and an annotation storage and feedback module. Through cooperative work of the two core modules, namely the rapid individual calibration module and the dynamic task scheduling module, the problems that decoding performance is reduced during cross-subject and cross-task switching caused by individual differences, and multi-task labeling efficiency is low due to task scheduling strategy stiffness are solved accurately. Data persistent storage, model online updating and user interaction feedback are completed through storage, optimization and feedback links, periodic retraining is carried out by utilizing daily successful labeling data through a model continuous learning link, and finally, a zero-threshold individual rapid adaptation and high-efficiency multi-task collaborative labeling target is achieved.

The invention provides a method for generating a personalized nerve regulation and control stimulation scheme, and aims to generate a precise nerve regulation and control stimulation scheme for an individual through an electroencephalogram evaluation result. The method comprises the following steps: firstly, constructing a scheme library containing a plurality of transcranial electrical stimulation basic schemes; secondly, collecting resting-state electroencephalogram signals of a user, extracting electroencephalogram characteristic indexes related to emotion, cognition and sleep, and comparing the electroencephalogram characteristic indexes with a norm database to judge cognition risks; generating a primary stimulation scheme according to an evaluation result, and if a cognitive risk exists, further detecting that a feature index is abnormal and generating a targeted correction scheme; all the schemes are subjected to priority ranking, and the sequence is determined according to the abnormal severity degree and the clinical weight; and finally, outputting a personalized treatment scheme sequence of one week. According to the method, the accuracy and effectiveness of treatment are improved, high individuation, systematicness, practicability and dynamic optimization potential are achieved, and powerful support is provided for nerve regulation and control treatment.

The present invention relates to the technical fields of human-computer interaction and brain informatics, and in particular to a closed-loop neurofeedback method and device based on haptic memory electroencephalogram signal conditioning. The method comprises: collecting electroencephalogram signals of a subject, and performing real-time online data preprocessing; designing a haptic memory stimulation task, activating the haptic memory of the subject, and recording a corresponding electroencephalogram signal response; performing feature extraction and analysis on a corresponding electroencephalogram signal to obtain an electroencephalogram feature related to haptic memory; and on the basis of a haptic memory electroencephalogram feature signal, designing a closed-loop neurofeedback system, the closed-loop neurofeedback system monitoring the electroencephalogram signals of the subject in real time, performing real-time stimulation on the basis of a preset haptic stimulation task, and recording a corresponding electroencephalogram response. In the technical solution of the present invention, the haptic memory electroencephalogram feature signal is combined with the closed-loop neurofeedback system, thus providing new ideas and methods for the diagnosis and treatment of haptic memory-related diseases.

The invention belongs to the field of flight safety, and discloses a civil aviation unit cognitive state difference evaluation method and system based on an LSTM-MHSA algorithm and a hierarchical coupling framework, and the method comprises the steps: S1, respectively collecting the electroencephalogram and psychological cognitive data of a pilot before and after the execution of a flight task, and after preprocessing, the scaled electroencephalogram features and psychological cognitive features are reconstructed to form a unified individual cognitive ability feature multivariate time sequence sample set. S2, constructing a deep learning model LSTM-MHSA, and extracting pilot personal cognitive ability time sequence evolution characteristics and pilot and copilot cognitive interaction fusion characteristics; s3, using an LSTM-MHSA model to evaluate the overall ability scores of the pilot in different psychological cognition dimensions and the psychological cognition cooperation mode of the pilot and the copilot in the flight mission period, and achieving the difference evaluation of the cognitive state of the civil aviation unit. The method aims at comprehensively evaluating the psychological cognitive ability of the pilot from the aspects of individuals and units.

The invention provides an emotion recognition method based on electroencephalogram feature fusion and a double-stage attention mechanism, and the method comprises the following steps: A, electroencephalogram signal processing: carrying out the preprocessing of an electroencephalogram signal; and B, double-stage attention feature fusion: in each selected frequency band, adopting a double-stage attention mechanism to fuse the electroencephalogram features, and generating fusion features for emotion classification. And C, double-branch feature extraction: performing double-branch 3D convolution processing on the fused features, extracting multi-scale space-spectral time features, and splicing the multi-scale space-spectral time features along a channel dimension to form uniform features. And D, classification and output: inputting the unified features into a classifier, and generating an emotion category prediction result through a flattening layer and a full connection layer. According to the method, the difference entropy, the power spectrum density and the difference entropy asymmetry feature are fused through unified three-dimensional feature representation, a double-stage attention mechanism is introduced, and high-accuracy emotion recognition is achieved.

The present invention specifically relates to a method and apparatus for tactile-motion electroencephalogram signal-based closed-loop neurofeedback training. The method comprises: collecting the current finger somatosensory temporal discrimination threshold and electroencephalogram data of a test individual; using a closed-loop neurofeedback system and an electroencephalogram amplifier to perform neurofeedback training on the test individual; collecting the current finger somatosensory temporal discrimination threshold and electroencephalogram data of the test individual again, and by means of processing behavioral data of the finger somatosensory temporal discrimination threshold of the test individual and electroencephalogram feature data of the test individual, acquiring an effect after the neurofeedback training. By utilizing the impairment of a primary somatosensory cortex to extract tactile electroencephalogram feature indicators that are less constrained by cognitive abilities, the present invention enables a patient, during the closed-loop neurofeedback training, to observe in real time signals of the patient's brain-related electroencephalogram feature indicators and perform self-regulation, thereby more effectively changing the patient's own basic neural mechanisms, and achieving the control and intervention of early Alzheimer's disease.

The invention belongs to the field of biomedical engineering, particularly relates to a multi-modal emotion recognition method based on electroencephalogram signal and facial expression fusion, and aims to improve the accuracy of emotion recognition. The method comprises the steps that electroencephalogram signals and a face video of a subject are collected, frequency band extraction, time window division and standardization processing and key frame extraction and face embedding coding are conducted respectively, time synchronization is achieved through data alignment, electroencephalogram time sequence fragments are input into a preset coding network to extract electroencephalogram time sequence features, and the electroencephalogram time sequence features are obtained. Inputting the facial spatial feature matrix into a preset convolution and time sequence fusion network to extract facial spatial and temporal features, then realizing dynamic interaction between modals through a cross attention mechanism to obtain preliminary fusion features, obtaining modal confidence degrees corresponding to the modals through a preset regression network, weighting the electroencephalogram feature matrix and the facial expression feature matrix, and obtaining an electroencephalogram feature matrix and a facial expression feature matrix; and outputting a final fusion feature matrix. And inputting the final fusion feature matrix into a preset classifier for emotion category prediction, and outputting an emotion recognition result.

The invention provides a mood disorder assessment system based on multi-level feature fusion, and the system comprises a data collection unit which is used for collecting electroencephalogram signals of a plurality of brain regions of a to-be-assessed patient; the electroencephalogram feature extraction unit is used for extracting electroencephalogram features corresponding to the electroencephalogram signals of the brain regions; the multi-level feature extraction unit is constructed on the basis of the symptom features of the multiple testees and the corresponding electroencephalogram features, and is used for performing multi-level electroencephalogram feature latent variable extraction on the electroencephalogram features of the brain regions of the patient to be evaluated to obtain multi-level electroencephalogram feature latent variables; and the feature fusion and classification unit is used for carrying out classification prediction based on the electroencephalogram feature latent variables to obtain a mood disorder assessment result of the patient to be assessed. The method solves the problem that a mood disorder assessment system in the prior art adopts a single feature extraction and learning strategy and has no constraint of symptom information, so that the recognition capability of a model for mood disorders of different functional abnormality types is limited.

The invention discloses an electroencephalogram feature recognition method for motion intention prediction in a lower limb alternate stepping process. The electroencephalogram feature recognition method comprises the following steps: acquiring an electroencephalogram signal of a subject before the subject executes an action under the requirement of a lower limb alternate stepping motion execution normal form; preprocessing the electroencephalogram signal to obtain an initial signal, performing label labeling on the initial signal according to an action type, and forming a data set by a label and the initial signal; constructing a neural network model, wherein the neural network model comprises a multi-scale grouping space-time convolution module, a self-attention mechanism module for updating external memory based on Top-k gating and a grouping causal time convolution module; and training the neural network model in a supervised manner by using the data set to obtain a prediction model for predicting the action type. According to the method provided by the invention, the movement intention of the patient can be recognized by the movement brain-computer interface before the lower limb alternate movement, the control of the rehabilitation robot on the specific lower limb movement by the brain intention is realized, and the method has an important value on the clinical application of a brain-controlled rehabilitation robot system.

The invention relates to the field of electroencephalogram signal processing and neural engineering, and discloses a multi-mode electroencephalogram feature fusion decoding method. The method comprises the following steps: carrying out noise reduction, calibration and standardization processing on an original electroencephalogram signal to generate a standard signal; extracting mu / beta rhythm time-frequency features and Hjorth time-domain features of the signals, and generating a fusion feature vector through PCA dimension reduction fusion; performing pre-classification and weight optimization by using an SVM classifier, performing space and time sequence feature extraction and classification through a CNN-LSTM network, and outputting a classification probability; model parameters are updated according to the probability and verified, and finally real-time control signals suitable for various communication interfaces are generated. According to the method, the accuracy and robustness of electroencephalogram signal decoding are improved, and efficient and self-adaptive motor imagery brain-computer interface control is realized.

The application provides a kind of active and passive intention recognition method and system based on electroencephalogram signal: the method comprises: obtaining the electroencephalogram signal of test personnel;Extract the electroencephalogram feature of multiple frequency bands based on electroencephalogram signal;And based on the electroencephalogram signal power spectrum density of preset frequency band in the electroencephalogram feature of multiple frequency bands, obtain active feature and passive feature;Multiple frequency bands of electroencephalogram feature, active feature and passive feature are input into pre-trained active and passive intention recognition model, and the active and passive intention categories of test personnel are obtained by identification;Active and passive intention recognition model is trained based on the electroencephalogram signal obtained from active intention experiment and passive intention experiment.The application solves the problem that the processing of electroencephalogram signal in the prior art does not consider the influence of active and passive intention, resulting in poor accuracy of control instruction output by brain-computer interface.

The invention provides a child burn dressing change pain management method and system based on tactile and auditory stimulation. The method comprises the following steps: constructing a personalized pain management model, and collecting multi-modal data of a patient; and extracting fusion multi-modal data features, and generating tactile and auditory stimulation parameters by using a reinforcement learning model after obtaining fusion features. And the parameters are applied to touch and hearing devices, and personalized stimulation is synchronously applied during dressing change. Cooperative regulation and control of tactile and auditory stimulation are realized by means of multi-modal data fusion and reinforcement learning models, that is, time domain, emotion and electroencephalogram features are deeply fused to generate fusion features, and tactile and auditory parameter joint action vectors are generated through an Actor network based on the fusion features. Through multi-mode information interaction, the tactile stimulation inhibits wound pain signal conduction, the auditory stimulation adjusts amygdaloid nucleus emotional response, the analgesia efficiency is improved through cooperation of the tactile stimulation and the auditory stimulation, and the tolerance threshold value of children to single stimulation is reduced.

The invention discloses an intelligent asthenopia detection method, system and device, and relates to the field of asthenopia detection.The method comprises the steps that feature extraction is conducted on electroencephalogram information and eye movement information to obtain an electroencephalogram feature set and an eye movement feature set; respectively inputting the electroencephalogram feature set and the eye movement feature set into a plurality of preset asthenopia detection models to obtain a plurality of asthenopia detection results; determining a final asthenopia detection result according to the plurality of asthenopia detection results; wherein different preset asthenopia detection models correspond to different machine learning models, and each preset asthenopia detection model is obtained by training the machine learning model according to the asthenopia detection sample set. According to the invention, real-time, efficient, objective and accurate asthenopia detection can be realized.

The invention provides an emotion recognition method, device and equipment based on electroencephalogram signals and a medium. The emotion recognition method based on the electroencephalogram signals comprises the steps that original electroencephalogram signals under all electrode positions are obtained; electroencephalogram characteristic data are extracted according to the original electroencephalogram signals; extracting frequency band and spatial features from the electroencephalogram feature data to obtain a space-frequency feature map; a bidirectional Mama network is adopted to extract time sequence characteristics of the space-frequency characteristic pattern in each time period, and a time sequence characteristic sequence is obtained; fusing each time sequence feature in the time sequence feature sequence on a time period dimension to obtain a fused time sequence feature; and recognizing the fusion time sequence features by using a feedforward neural network to obtain an emotion recognition result. According to the emotion recognition method and device based on the electroencephalogram signals, the equipment and the medium, deep feature fusion can be achieved, hidden features related to emotion changes in the electroencephalogram can be fully mined, and the accuracy of emotion recognition is improved.

The invention relates to the technical field of sleep environment intelligent regulation and control, and provides a sleep thermal comfort self-adaptive regulation and control method based on electroencephalogram nerve feedback. The method comprises the following steps: firstly, acquiring three electroencephalogram channel signals FP1, FP2 and FPz of the forehead of a user, and identifying a sleep stage through preprocessing and an automatic staging algorithm; then quantile alignment, robust processing and channel confidence weighted fusion are carried out on the electroencephalogram signals of all the channels, and comprehensive electroencephalogram features and multiple sets of electroencephalogram feature parameters are extracted; calculating a nerve thermal comfort comprehensive index NTx based on the characteristics, and realizing prediction of an individual thermal comfort score TSV by using an LSTM model; finally, a temperature control instruction is automatically generated according to the TSV and the sleep stage, linkage adjustment is conducted on an air conditioner and bed area temperature control equipment, and individualized and continuous sleep thermal comfort self-adaptive optimization is achieved.

Disclosed electroencephalogram (EEG) monitoring and detection systems and methods can detect differentiated electrographic seizure characteristics of different seizure types across a diverse group of patients. Disclosed systems and methods can utilize EEG data collected by discrete wireless EEG sensors positioned on a scalp of a patient.

The invention discloses a music teaching system based on artificial intelligence, and particularly relates to the technical field of artificial intelligence, and the music teaching system comprises a neurocognitive adaptation module, a music DNA map construction module, a cognitive load monitoring module, an anti-AI dependence adjustment module and a multi-source data fusion unit. According to the music teaching system based on artificial intelligence, a dynamically evolved personalized teaching model is constructed through multi-modal data fusion and real-time analysis driven by artificial intelligence. The neurocognitive adaptation module is combined with electroencephalogram feature analysis and physiological signal monitoring to accurately capture cognitive preferences and ability bottlenecks of the learner; the music DNA map is based on the quantum enhancement modeling technology, the skill development trajectory is continuously updated and predicted, the AI teaching strategy can realize millisecond-level dynamic adjustment according to the neural feedback and behavior data of the learner, the skill mastering efficiency and the knowledge retention rate are remarkably improved, and the limitation of staticization and simplification of a traditional teaching system is broken through.

The invention provides an anti-semantic interference training method and a semantic communication method based on electroencephalogram joint learning. The method comprises the following steps: applying semantic interference to an original image to generate a disturbance image; respectively inputting the original image and the disturbance image into an image encoder to extract image features, and inputting corresponding electroencephalogram signals into an electroencephalogram encoder to extract electroencephalogram features; and based on the original image features, the disturbance image features and the electroencephalogram features, performing comparative learning through a bidirectional InfoNCE loss function, and updating parameters of an image encoder and an electroencephalogram encoder. Joint comparative learning is carried out through the electroencephalogram signals and the image data, the attention of an image encoder to key information is enhanced by utilizing the characteristics of the electroencephalogram signals, and therefore the robustness of a semantic communication system when the semantic communication system is attacked by semantic noise is improved. According to the method, the semantic interference resistance of a semantic communication system is enhanced based on the natural robust feature of the electroencephalogram signal, and the method is suitable for the complex situation that diversified and multi-type semantic interference exists.

The application relates to a method and device for identifying oral actions, a storage medium and an electronic device, the method comprising: acquiring an electromyogram signal to perform short-time Fourier transform, acquiring electromyogram low-frequency features and electromyogram high-frequency features, constructing an electromyogram feature matrix, acquiring an electroencephalogram signal to perform continuous wavelet transform, acquiring electroencephalogram time-frequency band features, constructing an electroencephalogram feature matrix, mapping the feature matrices into the same feature space, obtaining electroencephalogram transformed features and electromyogram transformed features, performing mean zero and variance normalization operations on the spliced results, and obtaining a standardized feature matrix; based on the standardized feature matrix, a linear mapping matrix and a lightweight self-attention function, an adaptive weighted feature vector is obtained; the distances between the adaptive weighted feature vector and prototype feature vectors corresponding to various oral action categories pre-labeled are calculated, and the oral action category of the adaptive weighted feature vector is determined based on the calculated distances. The communication efficiency can be improved.

The invention relates to the technical field of biomedical engineering, in particular to a waking-up system based on multi-modal electroencephalogram characteristic dynamic evaluation and closed-loop regulation, which comprises a multi-modal electroencephalogram acquisition module for acquiring multi-modal original electroencephalogram signals; the multi-dimensional awakening related electroencephalogram feature fusion extraction module is used for extracting a multi-dimensional fusion feature vector; the consciousness level and waking-up response dynamic evaluation module is used for loading an off-line constructed and completed deep learning model, carrying out real-time dynamic evaluation and generating a waking-up response dynamic evaluation result; the personalized closed-loop awakening regulation and control decision module is used for generating and dynamically updating a closed-loop awakening regulation and control scheme by adopting an awakening personalized self-adaptive regulation and control algorithm; and the multi-modal awakening regulation and control execution and man-machine interaction module is used for executing multi-modal awakening nerve regulation and control output and providing a visual dynamic evaluation result, awakening regulation and control parameters and a man-machine interaction interface for medical personnel and family members of the patient. Therefore, the problems of single electroencephalogram acquisition mode, one-sided electroencephalogram feature extraction and the like in the prior art are solved.