A cross-domain electroencephalogram emotion recognition method and system with structural semantic representation unification

Abstract

The application relates to the technical field of electroencephalogram signal processing, and discloses a cross-domain electroencephalogram emotion recognition method and system with unified structure and semantic representation, which comprises the following steps: step 1, acquiring multiple-source electroencephalogram segments and metadata thereof; performing pretreatment and frequency band feature extraction, and generating a scalp topological structure view; step 2, constructing a spatial topological view and a frequency band structure view, inputting the double-view Manhattan frequency-space fusion encoder, and obtaining a unified structure representation; step 3, mapping emotion labels to a unified valence-waking emotion semantic space; determining the semantic coverage radius, semantic reliability, effective semantic distance between samples and credible weight for each sample; constructing a ranking multi-sample comparison constraint, training the encoder and the projection head; and step 4, inputting downstream electroencephalogram data into the trained encoder, extracting generalization features and outputting results. The application can improve the generalization stability and representation consistency of a model under different acquisition conditions and different label systems.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal processing technology, specifically to a cross-domain EEG emotion recognition method and system with unified structural semantic representation. Background Technology

[0002] EEG-based emotion recognition has significant applications in areas such as mental health assessment, intelligent human-computer interaction, and immersive content recommendation. However, most existing emotion recognition models are designed and validated for single datasets. When faced with target data from different acquisition devices and experimental paradigms, model performance often drops sharply due to significant differences in the number of channels, electrode layout, sampling protocols, and coverage between datasets. This cross-domain generalization challenge severely restricts the process of bringing EEG emotion recognition technology from the laboratory to real-world applications.

[0003] To mitigate cross-domain distribution shifts, researchers have attempted to introduce domain adaptation and transfer learning techniques. For example, Chinese patent CN111259761A extracts easily transferable emotional features using a transferable attention neural network, Chinese patent CN112690793A employs maximum mean difference and domain adversarial networks to alleviate distribution shifts, and Chinese patent CN114239652A utilizes clustering and adversarial partial domain adaptation to achieve shared category associations. However, these solutions primarily focus on feature distribution alignment or domain discrimination confusion. Fundamental input structure heterogeneity issues such as differences in the spatial structure of electrode layout, mismatches in the number of channels, and inconsistencies in coverage have not been uniformly addressed. The models still tend to learn dataset-specific structural pseudo-features rather than emotion-related neural representations.

[0004] Meanwhile, the diversity of emotion labeling systems further exacerbates the difficulty of cross-domain modeling—different datasets employ discrete emotion categories, binary classification, multi-class classification, or continuous valence-arousal ratings, respectively. Existing methods are typically designed under a single labeling system, making it difficult to establish a unified semantic metric compatible with multiple label definitions. Furthermore, the joint modeling of spatial topological relationships and frequency band structural relationships is also insufficient, making it difficult to effectively capture brain region-frequency band coupling features across datasets.

[0005] Therefore, there is an urgent need for a cross-domain EEG emotion recognition method that can simultaneously unify the input structure representation and the emotional semantic measurement, so as to improve the generalization stability and representation consistency of the model under different acquisition conditions and different labeling systems. Summary of the Invention

[0006] The present invention aims to provide a cross-domain EEG emotion recognition method and system with unified structural semantic representation, which can simultaneously unify the input structural expression and emotional semantic measurement, and improve the generalization stability and representation consistency of the model under different acquisition conditions and different labeling systems.

[0007] To achieve the above objectives, the present invention provides the following basic solution.

[0008] Option 1 A cross-domain EEG emotion recognition method with unified structural semantic representation includes the following steps: Step 1: Obtain multi-source EEG segments and their channel, sampling, source, and emotion tag metadata. The emotion tag metadata includes at least one of discrete category tags, binary category tags, multi-category tags, or continuous valence-arousal scores. Preprocess the multi-source EEG segments and extract their frequency band features. Project the frequency band features onto a unified two-dimensional scalp topology grid based on the standard electrode coordinates of the effective scalp channels and interpolate them to generate a scalp topology view. Step 2: Construct a spatial topology view and a frequency band structure view from the scalp topology view. The spatial topology view includes scalp effective area indication information and channel observation support information. The frequency band structure view is used to express the response relationship between frequency bands and scalp partitions. The spatial topology view and the frequency band structure view are respectively input into the two coding branches of the dual-view Manhattan frequency-space fusion encoder. After image block embedding, global representation label convergence, Manhattan distance visual coding and adaptive weighted fusion, a unified structural representation is obtained. Step 3: Map the emotion labels to a unified valence-arousal emotion semantic space to obtain the semantic center of each sample in the emotion semantic space; determine the semantic coverage radius and semantic reliability of each sample based on the label source, label granularity, clarity of category definition and cross-domain semantic consistency, and determine the effective semantic distance and credibility weight between samples accordingly. The structure is uniformly represented and mapped to projected features by a projection head. A ranking-based multi-sample contrast constraint is constructed based on the effective semantic distance and credible weights. The encoder and projection head are then trained. Step 4: Input the downstream EEG data into the trained encoder in the same way, extract generalization features, and output emotion classification results, valence-arousal regression results, or joint recognition results.

[0009] Option 2 A cross-domain EEG emotion recognition system with unified structural semantic representation is used to execute a cross-domain EEG emotion recognition method with unified structural semantic representation as described in Scheme 1; comprising: The multi-source EEG data foundation construction module is used to acquire multi-source EEG segments and their channel, sampling, source and emotion tag metadata, perform unified preprocessing on the multi-source EEG segments, project and interpolate multi-band features to a unified two-dimensional scalp topology grid based on the standard electrode coordinates of the effective scalp channels to generate a scalp topology view, and construct a spatial topology view and a frequency band structure view from the scalp topology view; The structural heterogeneous unified pre-training module is used to input the spatial topology view and the frequency band structure view into the dual-view Manhattan frequency-space fusion encoder, and generate a structural unified representation through image block embedding, global representation label convergence, Manhattan distance visual encoding and adaptive weighted fusion. The emotion semantic ranking and comparison learning module is used to map the emotion labels to a unified valence-arousal emotion semantic space to obtain the semantic center of each sample. Based on the label source, label granularity, clarity of category definition and cross-domain semantic consistency, it determines the semantic coverage radius and semantic reliability of each sample, and determines the effective semantic distance and credibility weight between samples accordingly. Based on the effective semantic distance and credibility weight, it constructs a ranking multi-sample comparison constraint and trains the encoder and projection head. The projection feature generation module is used to map and normalize the unified representation of the structure through a projection head to obtain projection features; The downstream generalization evaluation and recognition output module is used to input the downstream EEG data into the trained dual-view Manhattan frequency-space fusion encoder after processing by the multi-source EEG data basic construction module, extract generalization features, and output at least one of emotion classification results, valence-arousal regression results, or joint recognition results.

[0010] The working principle and advantages of this invention are as follows: This invention provides a cross-domain EEG emotion recognition method and system that unifies structural and semantic representations, enabling simultaneous unification of input structural expression and emotional semantic measurement, thereby improving the model's generalization stability and representation consistency under different acquisition conditions and labeling systems. The key points are: First, this invention fundamentally solves the heterogeneity problem of channel number and electrode layout in multi-source EEG data by generating a unified two-dimensional scalp topology view. Unlike existing technologies that attempt to design complex graph networks to adapt to different topologies, this invention transforms discrete and non-uniform electrode positions into a regular unified grid through interpolation. Furthermore, it introduces scalp effective area indication information to distinguish between areas inside and outside the effective scalp area, and introduces channel observation support information to characterize the degree of support of each grid from real electrode observations. During the encoding process, it adaptively reduces the influence of long-distance interpolation and low-coverage areas, enabling data from different acquisition devices to be uniformly represented.

[0011] Second, this invention constructs a dual-view coding architecture consisting of a spatial topological view and a frequency band structure view, achieving complementary modeling of the spatial distribution patterns of EEG signals and the frequency band-brain region coupling relationship. The spatial topological view preserves detailed spatial distribution information, while the frequency band structure view explicitly models the response relationship between frequency bands and brain regions by dividing the scalp grid into brain regions and aggregating features from each frequency band. The two branches perform image patch embedding and Manhattan distance visual encoding on their respective two-dimensional structural grids, respectively, and after parallel processing, adaptively weighted fusion is performed at the feature level. Compared to the single-view scheme, the dual-view design allows the model to simultaneously utilize spatial topological priors and frequency band-region response priors.

[0012] Third, this invention achieves a dynamic optimal combination of spatial topology representation and frequency band structure representation through an adaptive weighted fusion unit, enabling the model to adaptively adjust the contribution ratio of the two branches according to the structural characteristics of each sample. For samples with rich spatial information and high electrode coverage density, the spatial topology view receives higher weight; for samples with few electrodes and high spatial interpolation uncertainty, the frequency band structure view receives higher weight. This makes the generation process of unified structural representation sample-adaptive, which is superior to a simple averaging scheme with fixed weights.

[0013] Fourth, this invention establishes a unified emotional semantic measurement framework compatible with discrete category labels and continuous valence-arousal ratings. Addressing the long-neglected issue of using different labeling systems for different datasets, this scheme maps discrete emotional categories to semantic prototype positions in the valence-arousal space, and directly uses normalized continuous ratings as semantic centers. Furthermore, it introduces a semantic coverage radius to characterize label granularity and boundary ambiguity, and a semantic reliability to represent the credibility of cross-domain semantic alignment. By calculating effective semantic distance, label differences are transformed into measurable spatial distance relationships, enabling labels from different sources and with varying formats to be processed uniformly within the same semantic space.

[0014] Fifth, this invention employs a ranking-based multi-sample contrast constraint to replace the simple positive-negative sample distinction in traditional contrastive learning. This mechanism constructs a ranking candidate set based on the effective semantic distance for any anchored sample within a batch, ensuring that the ranking by projected feature similarity aligns with the ranking by effective semantic distance. Compared to traditional schemes that only require positive samples to have higher similarity than negative samples, the ranking constraint requires the model to learn a fine-grained metric relationship between emotions, rather than just class distinctions. This helps optimize model performance and maintain the orderliness of the feature space.

[0015] Sixth, this invention achieves a fundamental shift in feature aggregation criteria from dataset source to emotional semantics through the synergistic optimization of structurally unified representation and semantic ranking contrastive learning. Structurally unified encoding maps samples from different datasets to a common structural reference frame, providing a comparable input space for semantically contrastive learning; semantically ranking contrastive learning, in turn, guides the encoder to learn semantically consistent representations, reducing overfitting to the structure of specific datasets. These two elements form a positive feedback loop, ensuring that the learned generalized features possess both structural consistency and emotional semantic consistency. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of an embodiment of the cross-domain EEG emotion recognition method and system with unified structural semantic representation of the present invention; Figure 2 This is a schematic diagram of the overall framework for the structure-semantic unified learning of a cross-domain EEG emotion recognition method and system embodiment of the present invention, which unifies structure and semantic representation. Figure 3 This is a schematic diagram of the average feature similarity experimental results under different effective semantic distance groupings in a semantic ranking consistency verification experiment of an embodiment of the cross-domain EEG emotion recognition method and system with unified structural semantic representation of the present invention. Figure 4 This is a schematic diagram of the visualization results of the cross-domain generalization feature distribution obtained in the cross-domain generalization feature distribution visualization experiment of an embodiment of the cross-domain EEG emotion recognition method and system with unified structural semantic representation of the present invention. Detailed Implementation

[0017] The following detailed description illustrates the specific implementation methods: The basic implementation examples are as follows: Figure 1 and Figure 2 As shown: A cross-domain EEG emotion recognition method with unified structural semantic representation includes the following steps: Step 1: Obtain multi-source EEG segments and their channel, sampling, source, and emotion label metadata. The emotion label metadata includes at least one of discrete category labels, binary category labels, multi-category labels, or continuous valence-arousal scores. Preprocess the multi-source EEG segments and extract their frequency band features. Project and interpolate the frequency band features onto a unified two-dimensional scalp topology grid based on the standard electrode coordinates of the effective scalp channels to generate a scalp topology view.

[0018] Specifically, raw EEG signals and associated information from multiple EEG datasets are acquired. In this embodiment, the acquired datasets include, but are not limited to, SEED (62 channels, 1000Hz, three emotion labels: happy, sad, neutral), SEED-IV (62 channels, 1000Hz, four emotion labels: happy, sad, fear, neutral), FACED (32 channels, 250Hz, nine emotion labels: pleasure, motivation, happiness, tenderness, anger, nausea, fear, sadness, and neutral), DEAP (32 channels, 512Hz, continuous valence-arousal score 1~9), and DREAMER (14 channels, 128Hz, continuous valence-arousal score 1~5), etc. Optionally, a self-built dataset may also be included, collected according to the channel configuration of the aforementioned datasets.

[0019] During the data acquisition phase, for each sample in each dataset, the original EEG signal matrix, channel name list, sampling frequency, dataset source identifier, and emotion label are recorded.

[0020] In this specification, unless otherwise specified, "sample" refers to a preprocessed EEG time window segment and its corresponding emotion label; "source dataset" refers to one or more EEG emotion datasets that participate in upstream cross-domain pre-training; "target dataset" refers to EEG emotion datasets that do not participate in upstream cross-domain pre-training and are used for downstream generalization evaluation; and "batch" refers to a set of multiple samples input into the model in one training iteration.

[0021] The following unified preprocessing procedure was then performed on the above multi-source EEG data: (1) Based on the channel type, channel name and source identifier, the following types of channels are automatically removed: electrooculography channels (such as channels containing keywords such as EOG, HEOG, VEOG), electromyography channels (such as channels containing keywords such as EMG), reference channels (such as channels containing keywords such as REF, CMS, DRL), trigger channels (such as channels containing keywords such as TRIG, STI) and invalid channels (such as channels whose channel names cannot be identified or whose values ​​are all zero).

[0022] (2) Unify electrode names through alias mapping table.

[0023] (3) Resampling - Data with different sampling frequencies are resampled to the target sampling frequency. For datasets with original sampling frequencies greater than 256Hz (such as 1000Hz for SEED), low-pass filtering is used for downsampling; for datasets with original sampling frequencies less than 256Hz (such as 128Hz for DREAMER), linear interpolation is used for upsampling.

[0024] (4) Filtering process - A zero-phase Butterworth bandpass filter is used to filter the continuous signal.

[0025] (5) Artifact removal – Independent component analysis (ICA) is used on continuous signals to identify and remove eye movement artifacts and electromyography artifacts.

[0026] (6) Time Window Segmentation – Based on a preset time window length (5 seconds in this embodiment), the processed EEG signal is segmented into time window segments, and EEG signals with different sampling rates are converted into alignable time frames according to a unified time window; each time window is further divided into several time frames at 1 second intervals, serving as the basic unit for subsequent frequency band feature extraction. For data with different sampling rates, the number of sampling points contained within the time window is used as the segmentation unit. For datasets with continuous rating labels, each time window inherits the label value of its complete trial; for datasets with discrete rating labels, each time window inherits the category label of its segment. Adjacent time windows are overlapped to increase the number of training samples and maintain temporal continuity. For data with a segment length less than a complete time window, zero padding or direct discarding is used.

[0027] (7) Standardization processing - For each channel in each dataset, calculate its mean and standard deviation independently and perform Z-score standardization.

[0028] Furthermore, for each time frame signal after the above preprocessing, power characteristics of five preset frequency bands are extracted using short-time Fourier transform (STFT) or filter bank methods. The preset frequency bands include at least one of delta band, theta band, alpha band, beta band, and gamma band, and in this embodiment, they are selected as 1~4Hz, 4~8Hz, 8~14Hz, 14~31Hz, and 31~49Hz respectively.

[0029] The preferred filter bank method in this embodiment is as follows: for the signal of each channel c in each time frame, a fourth-order zero-phase Butterworth bandpass filter is used to extract the component of the b-th frequency band. The square of the power is taken as the time average within the time frame and then logarithmically compressed to obtain the characteristic value of the channel for that frequency band. ; In the formula, T is the number of sampling points within that time frame.

[0030] The frequency band characteristics include at least one of frequency band power, differential entropy, power spectral density, or time-frequency energy.

[0031] Furthermore, the scalp topology view is generated by projecting and interpolating the multi-band features of the effective scalp channels within each time frame to the unified two-dimensional scalp topology mesh. Specifically, it includes the following sub-steps: (1) Establish a standard electrode coordinate template.

[0032] Based on the international 10-20 system standard, a three-dimensional coordinate template containing all possible electrode positions is established. The three-dimensional coordinates (x, y, z) of each electrode are determined according to the relative proportions of the electrodes on the scalp, with the center of the head as the origin and the line connecting the root of the nose to the occipital protuberance as the baseline. For the two-dimensional scalp topology mesh used in this embodiment, the three-dimensional coordinates are mapped to a two-dimensional plane. The mapping method is to project each electrode position onto a two-dimensional unfolded diagram of the standard scalp template to obtain two-dimensional coordinates (u, v).

[0033] (2) Determine a unified two-dimensional scalp topology mesh. Set the spatial resolution of the unified two-dimensional scalp topology mesh. In this embodiment, the mesh size is set to H×W=64×64 pixels, which can achieve a balance between preserving scalp spatial details and controlling computational complexity. For higher precision applications, it can also be set to 128×128 pixels. The two-dimensional spatial range of the mesh covers the electrode distribution range of the standard scalp template and extends outward by a certain proportion (e.g., 10%), and the two-dimensional coordinates of the mesh are normalized to the interval [-1,1]×[-1,1].

[0034] (3) Determine the effective scalp region in the unified two-dimensional scalp topology grid based on the standard electrode coordinate template. Specifically, for each pixel position in the grid, calculate its distance to the nearest standard electrode. If the distance value is less than or equal to a preset distance threshold (e.g., 1.5 times the standard electrode spacing), it is marked as inside the effective scalp region; otherwise, it is marked as outside.

[0035] (4) For the multi-band EEG features in each time frame, the feature values ​​of the effective scalp channels are projected onto a unified two-dimensional scalp topology grid.

[0036] (5) Generate channel observation support information. For any pixel position p in the unified two-dimensional scalp topology mesh, calculate its channel observation support s(p) using the following formula: ; In the formula, Let s(p) be the projection position of the c-th effective scalp channel on the grid, where C is the number of effective scalp channels, h is the distance bandwidth, and Z is the maximum summation value of all pixels on the grid. Let s(p) be normalized to [0,1]. In this embodiment, h=0.15 is taken, and s(p)=0 is set outside the effective scalp region.

[0037] A larger s(p) indicates that the pixel is closer to the real electrode, the surrounding electrodes are more dense, and the interpolation is more reliable. Alternatively, a monotonically decreasing function of the distance from the pixel to the nearest effective channel, or the sum of its linear interpolation weights, can be used.

[0038] The channel observation support information in the spatial topology view is used to reduce the impact of features corresponding to low-support grids during spatial topology coding.

[0039] (6) Stack the feature maps of each frequency band along the frequency band dimension to form a multi-channel scalp topology view. Each pixel position in this view contains the interpolated feature value of each frequency band, representing the spectral feature distribution at that spatial position.

[0040] Step 2: Construct a spatial topology view and a frequency band structure view from the scalp topology view. The spatial topology view includes scalp effective area indication information and channel observation support information. The frequency band structure view is used to express the response relationship between frequency bands and scalp partitions.

[0041] Specifically, the spatial topology view is composed of the following information stitched together in the channel dimension: (1) feature maps of each frequency band (five frequency bands in this embodiment), i.e., the five channels of the above scalp topology view; (2) scalp effective area indication information as an independent binary channel; (3) channel observation support information as an independent continuous value channel.

[0042] In the subsequent spatial topological encoding process, the effective region indication information is used to guide the model's attention to focus only on feature interactions within the effective scalp region, forcing the attention weights of pixels outside the effective region to zero. Observational support information serves as the feature importance weight; for long-distance interpolation, missing channel completion, or sparse coverage locations, its observational support is lower than that for directly observed locations, reducing the impact of the corresponding grid features on the structural representation.

[0043] The frequency band structure view is formed by aggregating scalp spatial information along the partition dimension. The specific construction process is as follows: (1) The unified two-dimensional scalp topology grid is divided into several preset partitions according to standard brain regions. In this embodiment, based on the electrode distribution of the international 10-20 system and the functional anatomical partitions of the brain, the scalp grid is divided into the following partitions: frontal lobe, central lobe, temporal lobe, parietal lobe, and occipital lobe. The boundary of each partition is determined according to the Voronoi division of the standard electrode coordinates, that is, each grid pixel belongs to the brain region where the nearest standard electrode is located.

[0044] (2) For each partition r, aggregate the five frequency band features of all pixel locations within the partition to obtain the aggregated features. The aggregation method is a weighted average, with the weights being the observation support at each pixel location. For a partition where the sum of the observation support is zero (i.e., the partition has no effective electrode coverage), the aggregated features of that partition in each frequency band are set to zero, and the image block corresponding to that partition is marked as invalid in the self-attention calculation of the frequency band structure coding branch, forcing the attention weight pointing to that image block to be set to zero.

[0045] (3) Aggregate the features of each partition in each frequency band Arranged in a two-dimensional matrix format corresponding to "partition-band". In this embodiment, 5 partitions are used as rows and 5 bands as columns to form a 5×5 response matrix. The row order of the matrix is ​​fixed as [Front lobe F, Central lobe C, Temporal lobe T, Parietal lobe P, Occipital lobe O], and the column order is fixed as follows: Each element of the matrix The value represents the aggregated feature value of the r-th partition in the b-th frequency band, which characterizes the average activation intensity of that frequency band in that brain region.

[0046] Unlike spatial topological views, frequency band structure views discard fine spatial location information but retain macroscopic brain region spatial structures (i.e., the relative relationships between regions), while explicitly modeling the coupling relationship between frequency bands and brain regions. This "region-frequency band" matrix structure allows frequency band structure views to serve as complementary inputs to spatial topological views, both inputting into the dual-view encoder.

[0047] Furthermore, the spatial topology view and the frequency band structure view are respectively input into the two coding branches of the dual-view Manhattan frequency-space fusion encoder. After image block embedding, global representation label convergence, Manhattan distance visual coding and adaptive weighted fusion, a unified structural representation is obtained.

[0048] The dual-view Manhattan frequency-space fusion encoder includes a spatial topology coding branch, a frequency band structure coding branch, and an adaptive weighted fusion unit. The spatial topology coding branch and the frequency band structure coding branch perform image block division and image block embedding on their respective two-dimensional structure grids, and each adds a global representation marker.

[0049] Specifically, for the spatial topology view, it is first divided into non-overlapping image patches. Each image patch is flattened and mapped to a D-dimensional embedding vector through a learnable linear mapping layer (image patch embedding layer), where D is the hidden dimension of the model. After image patch embedding, the spatial topology view branch obtains the image patch feature sequence.

[0050] For the frequency band structure view, since the view size is small (5×5), in this embodiment, each "partition-frequency band" element is treated as an independent image block, and each image block actually contains one element (i.e., a single pixel block). Each image block is also mapped to a D-dimensional embedding vector through a linear mapping layer to obtain the image block feature sequence of the frequency band structure view.

[0051] Since the rows (partitions) and columns (bands) in the frequency band structure view have clear order semantics (partition order, frequency band frequency order), there is no need to add additional learnable positional encoding, as the order itself is the structural prior. In the subsequent Manhattan distance visual encoding, the two-dimensional structural position coordinates (u,v) of each image patch in this branch are taken as its row and column numbers in the "partition-frequency band" matrix.

[0052] In each of the two branch image patch feature sequences, a learnable global representation label ([CLS] label) is added. The global representation label is used in the subsequent Manhattan distance visual encoding process to aggregate global information from all image patches, and is finally used as the global representation output of that branch.

[0053] In the Manhattan distance visual coding, the Manhattan distance visual coding unit is used as the core component of the dual-view Manhattan frequency-space fusion encoder. Its structure is similar to the Transformer encoder layer, but Manhattan distance attenuation based on the two-dimensional structural position of the image block is incorporated into the self-attention calculation.

[0054] For any view branch, assume that the branch has N image patches (including global representation labels), where the first N... One is an image patch, and the last is a global representation marker. The positions of the m-th and n-th image patches in the two-dimensional structured mesh are denoted as follows: and The structural correlation weight between the two is determined by the following formula: ; In the formula, v is used to distinguish between the spatial topology view branch and the frequency band structure view branch; and Here, are the query vector and the key vector, respectively; d represents the feature dimension; and β represents the Manhattan distance decay coefficient, used to control the intensity of distance decay between image patches. The position of the l-th image patch in the two-dimensional grid is denoted as . .

[0055] For a location pair containing a global representation tag (i.e., m or n equals N), its Manhattan distance term is set to zero, allowing the global representation tag to interact freely with all image patches without being limited by spatial distance decay.

[0056] The value of the Manhattan distance attenuation coefficient β determines the strength of the model's preference for spatial proximity. The larger β is, the more the attention is focused on image patches that are close to the structural location.

[0057] In this embodiment, β is set separately according to the attention heads, so that each head in the same layer has a multi-scale receptive field from strong to weak: ; In the formula, H is the number of attention heads, h is the attention head number and takes values ​​from 0 to H-1, and ζ and ξ are the initial attenuation intensity and attenuation span parameters, respectively.

[0058] In this embodiment, H=8, ζ=1.0, and ξ=3.0, with the values ​​of the eight attention heads being approximately 0.69, 0.49, 0.35, 0.26, 0.19, 0.15, 0.11, and 0.08, respectively. The image patch positions of the spatial topology view branch correspond to the actual scalp spatial distances, using the aforementioned head-decreasing sequence. The row and column indices of the frequency band structure view branch only represent the proximity between the partition order and the frequency order, and this branch contains only 25 image patches. Excessive attenuation would inhibit cross-partition and cross-frequency band coupling modeling, so a relatively small fixed value β=0.10 is used.

[0059] After the above structural association weights are calculated, the output is calculated using the standard multi-head self-attention method.

[0060] Furthermore, each Manhattan distance visual coding unit also includes residual connections, layer normalization, and a feedforward network (FFN).

[0061] In this embodiment, the spatial topology coding branch and the frequency band structure coding branch each stack L layers of Manhattan distance visual coding units. Specifically, L=4, the number of attention heads per layer H=8, and the hidden dimension (image patch embedding dimension) D=128. The spatial topology branch divides the 64×64 pixel scalp grid into 64 image patches of 8×8 pixels, while the frequency band structure branch treats each element in the 5×5 partition-frequency band response matrix as a single-pixel image patch, for a total of 25 image patches. After passing through L layers of Manhattan distance visual coding units, the output vectors corresponding to the global representation labels are extracted from the two branches respectively. The global representation label output of the spatial topology coding branch is... The global representation tag output of the frequency band structure coding branch is .

[0062] The adaptive weighted fusion unit uses and As input, perform weighted fusion using the following formula: ; In the formula, For spatial global representation, For global frequency band characterization, Let w be the gating function, and b be the learnable parameters (where w is the learnable weight vector and b is the learnable bias scalar). Indicates feature splicing, For the unified structural representation of the i-th sample, The spatial topological representation fusion weights are for the i-th sample.

[0063] Fusion weights A value close to 1 indicates that the sample relies more on the representation of the spatial topology view, while a value close to 0 indicates that it relies more on the representation of the frequency band structure view. For example, for dataset samples with high electrode coverage density and rich spatial information, It tends to favor larger values; for samples with a small number of electrodes and high spatial interpolation uncertainty. It tends to favor smaller values. This fusion weight is learned automatically through end-to-end backpropagation, requiring no manual setting.

[0064] Step 3: Map the emotion labels to a unified valence-arousal emotion semantic space to obtain the semantic center of each sample in the emotion semantic space.

[0065] Specifically, the semantic mapping in the emotion semantic space is determined by the following formula: ; In the formula, This represents the original sentiment label of the i-th sample. To pre-define the semantic mapping relationship of emotions, It represents the semantic center composed of valence coordinates and arousal coordinates.

[0066] In this embodiment, the coordinates of the valence dimension and arousal dimension of the emotion semantic space are both normalized to [-1,1], and neutral emotions are located at the origin (0,0); a positive valence indicates a more positive emotion, and a positive arousal indicates a higher level of arousal.

[0067] For continuous valence-arousal scoring, let the scoring range be... The original valence and arousal score are respectively and The semantic center is obtained by linear normalization according to the following formula: .

[0068] For discrete category labels, binary category labels, or multi-category labels, a lookup table is pre-set based on the cyclic emotion model in emotion psychology, and the category name is used to look up the table and map it to the semantic prototype position in the emotion semantic space.

[0069] The semantic prototype positions used in this embodiment are shown in Table 1.

[0070] Table 1. Semantic Prototype Location Diagram

[0071] For datasets that only provide high / low binary classification labels, the high-valence class is mapped to (+0.50, 0.00), the low-valence class is mapped to (-0.50, 0.00), the high-arousal class is mapped to (0.00, +0.50), and the low-arousal class is mapped to (0.00, -0.50).

[0072] Categories with the same name are mapped to the same prototype position regardless of which dataset they come from, thereby achieving semantic alignment across datasets; the above mapping relationship can be supplemented or adjusted according to the actual labeling system used.

[0073] Furthermore, based on the source of the tags, the granularity of the tags, the clarity of the category definition, and the cross-domain semantic consistency, the semantic coverage radius is determined for each sample. and semantic reliability Based on the semantic center and semantic coverage radius of each sample, the effective semantic distance between samples is determined. And based on the semantic reliability, the credibility weight of each sample in participating in the ranking-based multi-sample comparison constraint is determined. The credibility weight is positively correlated with the semantic reliability.

[0074] Used to characterize the coverage of the tag in the valence-arousal space. A larger value indicates that the semantic boundaries of the tag are more ambiguous; Used to indicate the reliability of the semantic alignment of the sample. The closer the value is to 1, the more reliable the semantic tag. A decrease in the value indicates that the contribution of the sample to the comparison constraint should be reduced.

[0075] In this embodiment, the semantic coverage radius and the semantic reliability The specific value is determined according to the label attributes of the dataset to which the sample belongs, as shown in Table 2.

[0076] Table 2. Preset values ​​for semantic coverage radius and semantic reliability.

[0077] In other words, the coarser the label granularity and the more ambiguous the category definition, the larger the semantic coverage radius; the lower the credibility of the label source and the lower the cross-domain semantic consistency, the lower the semantic reliability. If the evoking paradigm or evoking material of a certain emotion category differs significantly in different datasets, causing the actual semantics of the same category to be not strictly equivalent, then the semantic coverage radius of the samples of that category is taken as 1.5 times the baseline value in the table above, and the semantic reliability is taken as 0.8 times.

[0078] The above values ​​can be adjusted according to the actual labeling system used, or they can be determined by statistical analysis of the intra-class standard deviation of continuous scoring and the consistency coefficient of labeling by multiple labelers.

[0079] Effective semantic distance between sample i and sample j and credible weight Calculate using the following formula: ; ; In the formula, The distance is Euclidean. The distance smoothing coefficient is set to 0.5 in this embodiment. and Let be the semantic coverage radius of sample i and the semantic coverage radius of sample j, respectively.

[0080] When the semantic coverage of two samples overlaps (i.e. When ), effective semantic distance This indicates that the two are semantically equivalent. When the semantic centers are far apart and their coverage areas do not overlap, It increases with the increase of semantic center distance. When the semantic reliability of any sample... or When the confidence weight is low The corresponding reduction.

[0081] Furthermore, the unified structural representation is mapped to projected features by the projection head, a ranking candidate set is constructed based on the effective semantic distance, and the contribution of each sample pair in the ranking candidate set is weighted according to the confidence weight to construct a ranking-based multi-sample comparison constraint, so that the ranking of projected feature similarity is consistent with the ranking of effective semantic distance, and the encoder and projection head are trained.

[0082] Specifically, the ranking-based multi-sample comparison constraint is constructed in the following manner: Unified representation of structure Input the projection head and obtain the projection features after normalization. : ; For any anchored sample i within a batch, construct a ranked candidate set based on the effective semantic distance: ; This set represents the set of all candidate samples k that have an effective semantic distance of not less than that of sample j relative to anchor sample i. In other words, if multiple samples have the same effective semantic distance as the anchor sample, these samples are grouped into the same group.

[0083] Based on this, a valence-arousal-guided contrastive loss for emotional semantic ranking is constructed: ; in, B is the projection head, and B is the number of samples in the training batch. For similarity function, For temperature coefficient, To prevent extremely small constants with a denominator of zero, in this embodiment, the similarity function is taken as the negative of the Euclidean distance between projected features, the temperature coefficient is taken as 1.0, and the extremely small constant is taken as... The number of samples B in the training batch is 288.

[0084] The physical meaning of this loss function is: if the effective semantic distance of sample j relative to anchor sample i is less than that of sample k (i.e., ... Then k will be included (in the middle), then the training process promotes Greater than In this way, the similarity ranking of projected features learned by the model is consistent with the effective semantic distance ranking in the emotion semantic space.

[0085] In this embodiment, the complete training process for upstream cross-domain pre-training is as follows: The first step is to construct the training dataset. From all the source datasets used in pre-training, samples are evenly distributed according to their origin to construct training batches. Even sampling means that the samples in each training batch are drawn as evenly as possible from all source datasets, avoiding model bias towards datasets with larger sample sizes.

[0086] The second step is to generate augmented views. For each EEG sample in the batch, two augmented views are generated. At least one augmentation method is randomly selected from the following options: (1) Time clipping: randomly clip 10% to 20% of the segments on the timeline and then interpolate them back to the original length.

[0087] (2) Frequency band perturbation: Add Gaussian noise with a standard deviation of 0.05 to the feature values ​​of five preset frequency bands, or randomly mask all features of a frequency band (set to zero).

[0088] (3) Random channel deactivation: randomly mask some channel features (set to zero) with a 10% probability.

[0089] (4) Topological interpolation perturbation: During the scalp topological interpolation process, 5% of the effective electrode position coordinates are randomly perturbed, and the perturbation amount does not exceed 20% of the distance between adjacent electrodes, in order to reproduce the small electrode positioning error between different datasets.

[0090] (5) Channel observation support perturbation: The channel observation support of 5% of the grid locations is randomly reduced by 30% to reproduce the situation of insufficient acquisition coverage.

[0091] (6) Gaussian noise perturbation: Add Gaussian noise with a standard deviation of 0.01 to the scalp topology view.

[0092] The two enhanced views share the same sentiment label and are treated as a positive sample pair in contrastive learning.

[0093] The third step is forward propagation. The spatial topology view and frequency band structure view of all samples in the batch (including the original samples and the enhanced view samples) are input into the dual-view Manhattan frequency-space fusion encoder to obtain a unified structural representation of each sample, and then the projection feature is obtained through the projection head.

[0094] The fourth step is to calculate the ranking-based contrastive loss. .

[0095] Step 5: Backpropagation and parameter update. The Adam optimizer is used for parameter update.

[0096] Step 6, Iterative Training. Repeat steps 1 through 5 until the model converges. The convergence condition is: the loss on the validation set no longer decreases after 10 consecutive epochs, or the total number of training epochs reaches a preset maximum value. The validation set consists of reserved samples from each source dataset and is independent of the training set.

[0097] Step 4: Input the downstream EEG data into the trained encoder in the same way (referring to the entire processing flow from the original EEG data to the input encoder in the aforementioned steps), extract the generalization features, and output the emotion classification result, valence-arousal regression result, or joint recognition result.

[0098] Specifically, emotion classification results refer to the discrete emotion category predictions output to downstream EEG samples. These predictions take the form of a probability distribution for each category or the category label corresponding to the highest probability.

[0099] Valence-Arousal Regression Results: These results predict continuous valence and arousal values ​​for downstream EEG samples. Specifically, they are regression outputs of two real values.

[0100] Joint recognition results: This refers to outputting both types of results simultaneously in the same output. For example, outputting both emotion category labels and valence-arousal value pairs at the same time, or using a multi-task learning approach where one branch performs classification and another branch performs regression. Its specific form varies depending on the downstream task settings.

[0101] The downstream EEG dataset is a dataset that was not used in training, or a dataset with different channel configurations, sampling protocols, or labeling systems than the source dataset. This step also includes downstream generalization evaluation. The downstream generalization evaluation includes at least one of the following evaluation methods: leave-one-out dataset testing, frozen encoder linear evaluation, and few-shot fine-tuning. Specifically: (1) Freeze encoder linearity evaluation: The pre-trained dual-view Manhattan frequency-space fusion encoder is frozen, and a single-layer linear classifier (or linear regressor) is trained on top of this feature using only the structural uniform representation of its output. The output dimension of the classifier is the number of classes in the target dataset, and the output dimension of the regressor is 2 (valence and wakefulness). The linear layer uses cross-entropy loss (classification) or mean squared error loss (regression) and is trained using the Adam optimizer.

[0102] (2) Fine-tuning with a small number of samples: A small number of samples (K samples per class, K=1, 5, 10) are randomly drawn from the training set of the target dataset. The last two layers of the pre-trained encoder are unfrozen and fine-tuned together with the classification head. A small learning rate is used during fine-tuning, and the training is performed for 30 epochs to prevent overfitting.

[0103] (3) Leave-one-out dataset test: Among the multiple datasets used for evaluation, one is sequentially excluded from upstream cross-domain pre-training. Pre-training is then performed using only the remaining datasets. The excluded dataset is then used as the target dataset for testing. This process is repeated for each dataset in turn. Since the excluded dataset is not accessed by the model during the pre-training phase, this method can verify the model's zero-shot generalization ability across datasets.

[0104] In this embodiment, the downstream evaluation uses the following metrics: balanced accuracy (suitable for imbalanced datasets), Kappa consistency coefficient (to measure the consistency between the classification result and the true label), and weighted F1 score (F1 score weighted by class support). For the continuous valence-awakening regression task, mean absolute error (MAE) and Pearson correlation coefficient (PCC) are used as evaluation metrics.

[0105] Through the above design, this invention forms a complete technical framework from upstream cross-domain pre-training to downstream generalization evaluation, possessing good scalability and deployment flexibility. In the upstream stage, the model can utilize multiple public datasets for unsupervised or weakly supervised pre-training; in the downstream stage, evaluation of new datasets can employ various protocols such as frozen encoder linear evaluation and few-shot fine-tuning, adapting to different labeled data availability conditions. Furthermore, the preset rules for semantic coverage radius and semantic reliability can be adjusted according to the actual dataset labeling system used, and parameters such as image patch size and grid resolution of the dual-view encoder can also be flexibly configured according to different application scenarios.

[0106] This embodiment also provides a cross-domain EEG emotion recognition system with unified structural semantic representation, used to perform the cross-domain EEG emotion recognition method with unified structural semantic representation as described above; including: The multi-source EEG data foundation construction module is used to acquire multi-source EEG segments and their channel, sampling, source and emotion tag metadata, perform unified preprocessing on the multi-source EEG segments, project and interpolate multi-band features to a unified two-dimensional scalp topology grid based on the standard electrode coordinates of the effective scalp channels to generate a scalp topology view, and construct a spatial topology view and a frequency band structure view from the scalp topology view; The structural heterogeneous unified pre-training module is used to input the spatial topology view and the frequency band structure view into the dual-view Manhattan frequency-space fusion encoder, and generate a structural unified representation through image block embedding, global representation label convergence, Manhattan distance visual encoding and adaptive weighted fusion. The emotion semantic ranking and comparison learning module is used to map the emotion labels to a unified valence-arousal emotion semantic space to obtain the semantic center of each sample. Based on the label source, label granularity, clarity of category definition and cross-domain semantic consistency, it determines the semantic coverage radius and semantic reliability of each sample, and determines the effective semantic distance and credibility weight between samples accordingly. Based on the effective semantic distance and credibility weight, it constructs a ranking multi-sample comparison constraint and trains the encoder and projection head. The projection feature generation module is used to map and normalize the unified representation of the structure through a projection head to obtain projection features; The downstream generalization evaluation and recognition output module is used to input the downstream EEG data into the trained dual-view Manhattan frequency-space fusion encoder after processing by the multi-source EEG data basic construction module, extract generalization features, and output at least one of emotion classification results, valence-arousal regression results, or joint recognition results.

[0107] To further demonstrate the superiority of this invention, the following comparative experiments are provided: (1) Cross-dataset migration experiment.

[0108] To verify the effectiveness of this invention in cross-domain EEG emotion recognition tasks, this embodiment designed a cross-dataset transfer experiment. The experiment selected the SEED dataset (62 channels, three emotion labels: happy, sad, and neutral) and the FACED dataset as the source and target domains, respectively. A leave-one-out cross-validation strategy was adopted to verify the generalization performance in the two transfer directions: SEED→FACED and FACED→SEED.

[0109] Traditional machine learning methods, convolutional neural network methods, and Transformer-type methods were selected as comparison methods. The balanced accuracy, Kappa consistency coefficient, and weighted F1 score were compared on two downstream datasets, FACED and SEED. The results are shown in Table 1, and the values ​​are expressed as mean / standard deviation.

[0110] Table 1 Results of cross-domain EEG emotion recognition experiment

[0111] As shown in Table 1, the proposed method demonstrates high overall performance on both datasets, significantly outperforming the three comparative methods. On the SEED dataset (FACED→SEED transfer direction), the proposed method achieves a balance accuracy of 63.86%, a 4.65 percentage point improvement over the best comparative method (Transformer class, 59.21%), with a Kappa coefficient improvement of 5.44 percentage points. On the FACED dataset (SEED→FACED transfer direction), the balance accuracy reaches 65.12%, a 10.94 percentage point improvement over the best comparative method (54.18%), with a Kappa coefficient improvement of 11.18 percentage points. The standard deviation of the proposed method (0.61–0.79) is significantly lower than that of the comparative methods (0.92–2.44), indicating better training stability and robustness.

[0112] It is noteworthy that on the FACED dataset, the number of target domain categories (9 categories) far exceeds that of the source domain (3 categories). The method of this invention performs exceptionally well in this scenario, validating that mapping discrete emotion categories to continuous semantic centers through the valence-arousal semantic space effectively alleviates the semantic gap caused by inconsistent label granularity. The improvement in the Kappa coefficient is particularly significant, indicating that the ranking-based multi-sample contrastive constraint maintains consistency between feature similarity ranking and semantic distance ranking, improving the consistency between classification results and true labels. These results demonstrate that this invention, through the synergistic effect of structurally heterogeneous unified encoding and emotion semantic ranking contrastive learning, enables the model to learn generalization features that simultaneously possess structural and semantic consistency, significantly improving the generalization performance of cross-domain EEG emotion recognition.

[0113] (2) Semantic ordering consistency verification experiment.

[0114] To verify whether the ranking-based multi-sample comparison constraint can maintain consistency between projected feature similarity ranking and semantic distance ranking, this experiment selected the FACED→SEED migration direction. After pre-training, projected features of all samples in the SEED dataset were extracted, and effective semantic distance and feature similarity were calculated for each sample pair. All sample pairs were divided into four groups (near distance, medium-near distance, medium-long distance, and long distance) according to their equal frequency from smallest to largest, and the average feature similarity of each group was calculated. A standard model (without Manhattan distance, ranking constraints, semantic coverage radius, or reliability) was used as a control.

[0115] Figure 3The comparison results of the two models are presented. The average feature similarity of the ordinary model decreases only slowly with increasing effective semantic distance, while the similarity in the far-distance group remains at a high level. The method of this invention, however, shows a clear and monotonous decreasing trend, with a significantly larger descent gradient than the ordinary model. The two curves intersect near the mid-distance range: in the near-distance group, the feature similarity of the method of this invention is significantly higher than that of the ordinary model, indicating that semantically similar samples are more closely grouped; in the far-distance group, it is significantly lower than that of the ordinary model, indicating that semantically dissimilar samples are more widely spaced. The results show that this invention successfully maintains the continuous proximity relationship (i.e., order preservation) of the emotional semantic space in the feature space through ranking-based contrast constraints. Semantically similar samples are closer in the feature space, and samples with large semantic differences are easier to distinguish.

[0116] (3) Visualization experiment of cross-domain generalization feature distribution.

[0117] To visually demonstrate the differences in feature distribution across different scenarios, this experiment employs the Uniform Manifold Approximation and Projection Method (UMAP) to reduce the projected features to two dimensions for visualization. After pre-training, projected features are extracted from the FACED and SEED datasets. Emotion categories are identified by dot-shaped fill patterns (hollow circles: negative, crosses: neutral, solid circles: positive), and dataset sources are identified by dot-shaped contour shapes (circles: FACED, triangles: SEED). Simultaneously, emotion clustering and dataset separation are calculated. These are contour coefficients calculated on the two-dimensional feature using emotion category and dataset source as cluster labels, respectively. A larger former indicates a more compact cluster within the same emotion category, while a latter closer to zero indicates a more thorough mixing of samples from different sources.

[0118] Figure 4 The comparison results are shown. In the ordinary model, FACED (circular contour) and SEED (triangular contour) samples are clearly separated in two-dimensional space, forming their own independent clusters. Samples of the same emotion category lack spatial alignment across datasets—the dataset separation is high while the emotion clustering is low, indicating that the feature distribution is more easily affected by differences in the dataset source.

[0119] In this invention, samples with the same filling method form relatively compact clusters in space, while samples from different datasets are intermingled within the same emotion region (e.g., circular and triangular contours of sample points are interspersed within a positive region). The emotion clustering degree is 0.46, and the dataset separation degree is only 0.02. The distribution order of the three emotion regions (negative, neutral, and positive) matches the semantic continuous change pattern of the valence dimension, verifying that ranking contrastive learning can transfer the continuous structure of the semantic space to the feature space. The above visualization results corroborate the quantitative experiments in Table 1, demonstrating that this invention, through the synergistic effect of structural heterogeneous unified encoding and emotion semantic ranking contrastive learning, transforms the clustering basis of the feature space from the dataset source to emotion semantics, enhancing the consistency of emotion semantics and reducing the impact of differences in dataset sources, thereby significantly improving cross-domain generalization performance.

[0120] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A cross-domain EEG emotion recognition method with unified structural semantic representation, characterized in that, Includes the following steps: Step 1: Obtain multi-source EEG segments and their channel, sampling, source, and emotion tag metadata. The emotion tag metadata includes at least one of discrete category tags, binary category tags, multi-category tags, or continuous valence-arousal scores. Preprocess the multi-source EEG segments and extract their frequency band features. Project the frequency band features onto a unified two-dimensional scalp topology grid based on the standard electrode coordinates of the effective scalp channels and interpolate them to generate a scalp topology view. Step 2: Construct a spatial topology view and a frequency band structure view from the scalp topology view. The spatial topology view includes scalp effective area indication information and channel observation support information. The frequency band structure view is used to express the response relationship between frequency bands and scalp partitions. The spatial topology view and the frequency band structure view are respectively input into the two coding branches of the dual-view Manhattan frequency-space fusion encoder. After image block embedding, global representation label convergence, Manhattan distance visual coding and adaptive weighted fusion, a unified structural representation is obtained. Step 3: Map the emotion labels to a unified valence-arousal emotion semantic space to obtain the semantic center of each sample in the emotion semantic space; determine the semantic coverage radius and semantic reliability of each sample based on the label source, label granularity, clarity of category definition and cross-domain semantic consistency, and determine the effective semantic distance and credibility weight between samples accordingly. The structure is uniformly represented and mapped to projected features by a projection head. A ranking-based multi-sample contrast constraint is constructed based on the effective semantic distance and credible weights. The encoder and projection head are then trained. Step 4: Input the downstream EEG data into the trained encoder in the same way, extract generalization features, and output emotion classification results, valence-arousal regression results, or joint recognition results.

2. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 1, characterized in that, The preprocessing includes, in sequence: resampling, filtering, artifact removal, time window segmentation, and normalization. This also includes: removing electrooculography (EOG) channels, electromyography (EMG) channels, reference channels, trigger channels, and invalid channels based on channel type, channel name, and source identifier; Unify electrode names through an alias mapping table; Based on a unified time window, EEG signals with different sampling rates are converted into alignable time frames; The frequency band characteristics include at least one of frequency band power, differential entropy, power spectral density, or time-frequency energy.

3. The cross-domain EEG emotion recognition method based on unified structural semantic representation according to claim 1, characterized in that, The scalp topology view is generated by projecting and interpolating the multi-band features of the effective scalp channels in each time frame to the unified two-dimensional scalp topology grid. The channel observation support information in the spatial topology view is used to reduce the impact of features corresponding to low-support grids during spatial topology coding.

4. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 1, characterized in that, The dual-view Manhattan frequency-space fusion encoder includes a spatial topology coding branch, a frequency band structure coding branch, and an adaptive weighted fusion unit. The spatial topology coding branch and the frequency band structure coding branch perform image block division and image block embedding on their respective two-dimensional structure grids, and each adds a global representation marker.

5. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 4, characterized in that, In the Manhattan distance visual encoding, for any view branch, the positions of the m-th image patch and the n-th image patch in the two-dimensional structured grid are respectively denoted as... and The structural correlation weight between the two is determined by the following formula: ； In the formula, v is used to distinguish between the spatial topology view branch and the frequency band structure view branch; and Here, are the query vector and the key vector, respectively; d represents the feature dimension; β represents the Manhattan distance decay coefficient, used to control the intensity of distance decay between image patches; the position of the l-th image patch in the two-dimensional grid is denoted as . .

6. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 4, characterized in that, The adaptive weighted fusion unit performs weighted fusion using the following formula: ； In the formula, For a global spatial representation, For global frequency band characterization, Let w and b be the gating function, and w and b be the learnable parameters. Indicates feature splicing, For the unified structural representation of the i-th sample, The spatial topological representation fusion weights are for the i-th sample.

7. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 1, characterized in that, The semantic mapping in the emotion semantic space is determined by the following formula: ； In the formula, This represents the original sentiment label of the i-th sample. To pre-define the semantic mapping relationship of emotions, It represents the semantic center composed of valence coordinates and arousal coordinates; For continuous valence-arousal scores, the normalized result is used as the semantic center; For discrete category labels, binary category labels, or multi-category labels, they are mapped to semantic prototype positions in the emotion semantic space.

8. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 7, characterized in that, Effective semantic distance between sample i and sample j and credible weight Calculate using the following formula: ； ； In the formula, This is the distance smoothing coefficient. and Let be the semantic coverage radius of sample i and the semantic coverage radius of sample j, respectively.

9. The cross-domain EEG emotion recognition method with unified structural semantic representation according to claim 8, characterized in that, The ranking-based multi-sample comparison constraint is constructed in the following way: Unified representation of structure Input the projection head and obtain the projection features after normalization. : ； For any anchored sample i within a batch, construct a candidate set for ranking: ； Constructing a ranking-based contrastive loss: ； in, B represents the projection head, and B represents the number of samples in the training batch. For similarity function, For temperature coefficient, To prevent constants with a denominator of zero.

10. A cross-domain EEG emotion recognition system with unified structural semantic representation, characterized in that, A method for performing a cross-domain EEG emotion recognition method with unified structural semantic representation as described in any one of claims 1-9; comprising: The multi-source EEG data foundation construction module is used to acquire multi-source EEG segments and their channel, sampling, source and emotion tag metadata, perform unified preprocessing on the multi-source EEG segments, project and interpolate multi-band features to a unified two-dimensional scalp topology grid based on the standard electrode coordinates of the effective scalp channels to generate a scalp topology view, and construct a spatial topology view and a frequency band structure view from the scalp topology view; The structural heterogeneous unified pre-training module is used to input the spatial topology view and the frequency band structure view into the dual-view Manhattan frequency-space fusion encoder, and generate a structural unified representation through image block embedding, global representation label convergence, Manhattan distance visual encoding and adaptive weighted fusion. The emotion semantic ranking and comparison learning module is used to map the emotion labels to a unified valence-arousal emotion semantic space to obtain the semantic center of each sample. Based on the label source, label granularity, clarity of category definition and cross-domain semantic consistency, it determines the semantic coverage radius and semantic reliability of each sample, and determines the effective semantic distance and credibility weight between samples accordingly. Based on the effective semantic distance and credibility weight, it constructs a ranking multi-sample comparison constraint and trains the encoder and projection head. The projection feature generation module is used to map and normalize the unified representation of the structure through a projection head to obtain projection features; The downstream generalization evaluation and recognition output module is used to input the downstream EEG data into the trained dual-view Manhattan frequency-space fusion encoder after processing by the multi-source EEG data basic construction module, extract generalization features, and output at least one of emotion classification results, valence-arousal regression results, or joint recognition results.

Images