A method for constructing an emotion recognition model, an emotion recognition method and a system

Abstract

The present invention provides a method for constructing an emotion recognition model, an emotion recognition method and a system, and belongs to the field of emotion recognition. The model construction method first collects 62-lead EEG signals containing different emotions within a predetermined time period, removes invalid bands after denoising, and then performs data segmentation according to the start and end points of the time when the emotions are generated, and retains the valid EEG signal fragments between the start and end points as samples; extracts the first positive sample subset based on the time nearest neighbor method; extracts the power spectrum density of five bands in each sample, calculates the differential entropy features, and constructs node feature representations, and extracts the second positive sample subset based on the node feature representation; then establishes a brain network diagram, calculates the phase-locked values ​​between the leads in each sample, obtains the adjacency matrix of the brain network diagram corresponding to the current sample, and constructs a topological structure feature representation, and then extracts the third positive sample subset, takes the union to obtain the final positive and negative sample set, and obtains a mature emotion recognition model after training.

Description

Technical Field

[0001] The present invention belongs to the field of emotion recognition, and specifically relates to a method for constructing an emotion recognition model, an emotion recognition method and a system. Background Art

[0002] The generation and expression of human emotions are closely related to physiological signals such as EEG, EMG, and heart rate. Therefore, by identifying and extracting the above signals, we can obtain accurate and parameterized emotional representations, thereby identifying emotions with high temporal resolution and objective evaluation.

[0003] In the prior art, in the field of emotion recognition based on electroencephalogram (EEG) signals, supervised learning methods are generally used. However, supervised learning methods not only rely on operators to manually label EEG signals under different emotions, which is a huge workload, but also require operators to have complete professional knowledge and be proficient in the operation process. Therefore, supervised learning methods are costly and difficult, highly professional, and difficult to implement. Summary of the invention

[0004] The present invention proposes a method for constructing an emotion recognition model, an emotion recognition method and a system. Different from the traditional supervised learning method that requires a large amount of labeled data, on the one hand, it ensures the low-cost acquisition of a large amount of training data, and on the other hand, it can fully mine the information carried by the electroencephalogram signal itself. At the same time, it improves the traditional contrastive learning by enhancing the data of the anchor sample to obtain the rough positive sample screening method of the positive sample, adopts three positive sample sampling strategies of time proximity, node feature similarity and brain network topological structure similarity, sets multiple positive samples for contrastive learning, and effectively improves the quality of the contrastive learning representation, so as to ensure that a good emotion recognition effect can be obtained in the end.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] In a first aspect, an embodiment of the present invention provides a method for constructing an emotion recognition model, comprising the following steps:

[0007] Step S1, collecting a number of 62-lead EEG signals within a predetermined time period, wherein the subject generates two or more different types of emotions within the predetermined time period, and recording the start and end points of the generation of the emotions;

[0008] Step S2, filtering the collected 62-lead EEG signals after denoising, retaining the effective bands in the EEG signals; and dividing the effective bands according to the start and end points of the time when the emotions are generated, retaining the effective EEG signal segments between the start and end points;

[0009] Step S3, among all valid EEG signal segment samples, positive samples are screened based on temporal proximity to obtain a first positive sample subset;

[0010] Step S4, in each valid EEG signal segment, extract the five bands of Delta wave, Theta wave, Alpha wave, Beta wave and Gamma wave contained in each lead EEG signal; extract the power spectrum density of the five bands respectively, and calculate the differential entropy features of the five bands respectively according to the power spectrum density, and then construct the node feature representation of the current segment according to the differential entropy features of the five bands;

[0011] Step S5, among all valid EEG signal segment samples, positive samples are screened based on the similarity of node feature representations to obtain a second positive sample subset;

[0012] Step S6, constructing a brain network graph G based on the valid EEG signal segments: G = (V, E); where V = {V1, V2, ..., V 62} is a set of vertices corresponding to the 62 leads; E is an edge connecting the vertices, and whether an edge between two vertices exists is determined according to the phase-locked value; the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment is calculated;

[0013] Among them, the phase lock value PLV ij It is used to measure the connectivity between leads. The calculation formula is:

[0014]

[0015] In formula (7), is the phase difference between the two lead signals i and j, t represents time, ω represents angular frequency, and N is the number of sampling points in the time range from the start time point StartNum to the end time point EndNum of the valid EEG signal segment;

[0016] Compare the phase-locking value between the leads with the preset threshold δ. If the phase-locking value is greater than or equal to the threshold δ, it is determined that there is a connection relationship between the brain regions corresponding to the two leads, and there is an edge E between the two vertices. If the phase-locking value is less than the threshold δ, it is determined that there is no connection relationship between the brain regions corresponding to the two leads, and there is no edge E between the two vertices.

[0017] Step S7, constructing a topological structure feature representation of each brain network graph G according to the adjacency matrix A;

[0018] Step S8, using valid EEG signal segments as samples, performing positive sample screening based on the similarity of the topological structure feature representation of the brain network graph G to obtain a third positive sample subset;

[0019] Step S9, taking the union of the first positive sample subset, the second positive sample subset and the third positive sample subset to obtain a final positive sample set, and the remaining samples are negative samples; the convolutional neural network is trained by the final positive sample set and negative sample set; a classification layer is added to the pre-trained convolutional neural network, a small amount of labeled sample data is randomly selected, and the classification layer is trained so that the parameters of the classification layer are adapted to the emotion recognition task, and an emotion recognition model is obtained.

[0020] As a preferred embodiment of the present invention, in the step S2, the denoising is performed by separating the collected original EEG signal into various independent components in the EEG signal using the independent component analysis method ICA, and removing various artifacts mixed in the EEG signal;

[0021] The formula for independent component analysis ICA when eliminating artifacts is as follows:

[0022] X=WS (1)

[0023] In formula (1), X represents the mixed signal of EEG signal and artifact, that is, the original signal collected directly from the scalp, W represents the ICA unmixing matrix, and S represents various independent signals.

[0024] As a preferred embodiment of the present invention, the specific process of step S3 is as follows:

[0025] Step S31, randomly select an anchor sample x a ; Define a hyperparameter T to represent the size of the constructed time window. In the anchor sample x a Define a time window of length 2T around in Represents the anchor sample x a The corresponding time;

[0026] Step S32: based on all samples x except the anchor sample i The sample selection constraints are constructed as follows:

[0027]

[0028] and:

[0029]

[0030] In formula (2), is the sample x i Is it in the time window? Determined by i is the sample label value, and the value 1 represents the constraint condition Q i Sample x in i is a positive sample, and the value is -1, which represents the constraint condition Qi Sample x in i is a negative sample.

[0031] As a preferred embodiment of the present invention, the average power spectral density PSD of each band in the five bands of each pilot signal in step S4 is expressed as:

[0032] PSD={∑[abs(FFTDATA i )]} / (EndNum-StartNum+1) (3)

[0033] In formula (3), FFTDATA i Refers to the signal value corresponding to the i-th time point in the valid EEG signal segment. EndNum is the end time point of the valid EEG signal segment, and StartNum is the start time point of the valid EEG signal segment;

[0034] Calculate the power spectrum density corresponding to the five bands of each lead signal, and approximate the differential entropy feature DE of the corresponding band under the current lead based on the power spectrum density:

[0035] DE=log2(PSD) (4)

[0036] The differential entropy eigenvalues ​​corresponding to the five bands of each guide signal are combined into a matrix of size 62×5 as the node feature representation of the current valid EEG signal segment:

[0037]

[0038] In formula (5), the matrix H is the node feature representation of the valid EEG signal segment.

[0039] As a preferred embodiment of the present invention, the positive sample screening is performed based on the similarity of node feature representation, and the specific process is as follows:

[0040] Step S51, calculate the differential entropy matrix H of the anchor samples respectively a And the differential entropy matrix H of the remaining samples i , i∈{1,2,...,batch size-1}, calculate the Mahalanobis distance between two matrices:

[0041]

[0042] In formula (6), Represents the anchor sample differential entropy matrix H a The value of the element in row j and column k, Represents the differential entropy matrix H of the i-th sample i The value of the element in the jth row and the kth column, Σ -1is the inverse matrix of the covariance matrix of the two differential entropy matrices;

[0043] Step S52, after calculating the Mahalanobis distance, the obtained (batch size-1) Mahalanobis distances are arranged in order from small to large, the proportional coefficient k1 is set, the first (batch size×k1) are selected as positive samples to form the second positive sample subset, and the remaining samples are used as negative samples.

[0044] As a preferred embodiment of the present invention, the calculation of the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment specifically includes:

[0045] For the brain network graph G, define the adjacency matrix of the graph as A. For the element a in A ij , calculated as follows:

[0046]

[0047] For the brain network graph G, its adjacency matrix is ​​a 62×62 symmetric matrix, expressed as follows:

[0048]

[0049] In formula (9), matrix A is the adjacency matrix corresponding to the brain network graph G corresponding to the valid EEG signal segment.

[0050] As a preferred embodiment of the present invention, the topological structure feature representation includes three factors, namely, average path length, clustering coefficient, and connection rate; wherein,

[0051] The average path length is defined as formula (10), which is used to characterize the length of the communication link between nodes as a whole:

[0052]

[0053] In formula (10), N V represents the number of nodes in the network, d ij Represents the length of the shortest path among all paths connecting nodes (i, j);

[0054] The clustering coefficient is defined as formula (11), which is used to represent the ratio of the number of connections between the nearest neighbors of a node to the maximum possible number of connections:

[0055]

[0056] and:

[0057]

[0058] In formula (11), N V represents the number of nodes in the network, ki Indicates the number of nodes directly connected to node i in the network, N Ei These k i The number of edges that actually exist between nodes;

[0059] The connection rate is defined as formula (12), which is used to measure N V The strength of the connection between nodes:

[0060]

[0061] In formula (12), N V Represents the number of nodes in the network, N E Represents the number of edges in the network;

[0062] The topological structure characteristic represents λ=(L glob (G), C (G), CR (G)).

[0063] As a preferred embodiment of the present invention, the positive sample screening is performed based on the similarity of the topological structure feature representation of the brain network graph G, and the specific process is as follows:

[0064] Step S81, randomly selected anchor sample x a , respectively find the anchor sample x a Brain network graph G a The corresponding adjacency matrix A a And the remaining samples x i Brain network graph G i The corresponding adjacency matrix A i ,i∈{1,2,...,batch size-1}.

[0065] The anchor sample x is calculated by equations (10)-(12) respectively. a The topological structure characteristics of a =(L glob (G a ),C(G a ),CR(G a )) and the remaining samples x i The topological structure characteristics of i =(L glob (G i ),C(G i ),CR(G i ));

[0066] Compute the cosine distance between two topological feature representations:

[0067]

[0068] In formula (13), represents the value of the jth element of the topological structure feature representation of the anchor sample, Represents the value of the jth element represented by the topological structure feature of the i-th sample;

[0069] Step S82, after calculating the cosine distance, arrange the obtained (batch size-1) cosine distances in order from small to large, set the proportional coefficient k2, select the first (batch size×k2) as positive samples to form the third positive sample subset, and the remaining samples as negative samples.

[0070] In a second aspect, an embodiment of the present invention further provides an emotion recognition method, the method comprising:

[0071] Obtaining valid EEG signal segments corresponding to the emotion to be identified; firstly collecting the EEG signal of at least one lead in the time period corresponding to the emotion, denoising the EEG signal to obtain a pure EEG signal, and then filtering to obtain the valid EEG signal segment; and extracting five wavebands of Delta wave, Theta wave, Alpha wave, Beta wave and Gamma wave from the valid EEG signal segment;

[0072] The valid EEG signal fragment is input into the emotion recognition model constructed as above, and the recognized emotion category is output.

[0073] In a third aspect, an embodiment of the present invention further provides an emotion recognition system, the system comprising: a 62-lead EEG signal acquisition module, a sample extraction module, a first positive sample screening module, a second positive sample screening module, a third positive sample screening module, a sample set synthesis module, a recognition model training module, and a recognition result output module; wherein,

[0074] The 62-lead EEG signal acquisition module is used to collect a number of 62-lead EEG signals within a predetermined time period, in which the subject generates two or more different types of emotions, and records the start and end points of the emotions as basic data of the sample; and is also used to collect at least one lead EEG signal corresponding to the emotion to be identified;

[0075] The sample extraction module is used to filter the collected 62-lead EEG signals after denoising, retaining the effective bands in the EEG signals; and to divide the effective bands according to the start and end points of the time when the emotions are generated, retaining the effective EEG signal segments between the start and end points;

[0076] The first positive sample screening module is used to screen positive samples based on temporal proximity among all valid EEG signal segment samples to obtain a first positive sample subset;

[0077] The second positive sample screening module is used to extract the five wave bands of Delta wave, Theta wave, Alpha wave, Beta wave and Gamma wave contained in each lead EEG signal in each valid EEG signal segment; extract the power spectrum density of the five wave bands respectively, and calculate the differential entropy features of the five wave bands respectively according to the power spectrum density, and then construct the node feature representation of the current segment according to the differential entropy features of the five wave bands; and is also used to screen positive samples based on the similarity of the node feature representation to obtain a second positive sample subset;

[0078] The third positive sample screening module is used to construct a brain network graph G based on valid EEG signal segments: G = (V, E); wherein V = {V1, V2, ..., V 62} is a set of vertices corresponding to the 62 leads; E is an edge connecting the vertices, and the existence of an edge between two vertices is determined according to the phase-locked value, and the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment is calculated, and the topological structure feature representation of each brain network graph is constructed according to the adjacency matrix A; it is also used to screen positive samples based on the similarity of the topological structure feature representation of the brain network graph G to obtain a third positive sample subset;

[0079] The sample set synthesis module is used to take the union of the first positive sample subset, the second positive sample subset and the third positive sample subset to obtain a final positive sample set, and the remaining samples are negative samples;

[0080] The recognition model training module is used to train the convolutional neural network through the final positive sample set and negative sample set obtained, and use the output of the convolutional neural network as the representation of the positive and negative samples, and substitute it into the loss function for calculation; when the loss function value is less than the set threshold, the training of the convolutional neural network is completed; a classification layer is added to the convolutional neural network obtained by pre-training, a small amount of labeled data is randomly selected, and the classification layer is trained so that the parameters of the classification layer are adapted to the emotion recognition task, and a mature emotion recognition model is obtained;

[0081] The recognition result output module is used to input the valid EEG signal segments corresponding to the emotion to be recognized into the emotion recognition model and output the emotion recognition result.

[0082] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0083] The emotion recognition model construction method, emotion recognition method and system provided in this embodiment are based on a self-supervised representation learning method. For the collected EEG signals representing emotions, three positive sample mining strategies are used to obtain positive and negative sample sets. This not only ensures the low-cost acquisition of a large amount of training data, but also improves the quality of contrastive learning representation. It can fully mine the information carried by the EEG signal itself, thereby effectively improving the accuracy and precision of the model and effectively improving the accuracy of emotion recognition.

[0084] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0086] Figure 1 A flow chart of a method for constructing an emotion recognition model based on EEG signals provided in an embodiment of the present invention;

[0087] Figure 2 Schematic diagram of the process of collecting and characterizing EEG signals in the construction method described in an embodiment of the present invention. DETAILED DESCRIPTION

[0088] After discovering the above problems, the inventors of this application conducted an in-depth study of existing emotion recognition methods. The study found that in addition to using supervised learning methods for emotion recognition, self-supervised learning methods can also be used for emotion recognition. Self-supervised learning can effectively solve the serious reliance of supervised learning methods on manually labeled data. Self-supervised learning does not require manually labeled data. It mainly uses auxiliary tasks to artificially construct pseudo-labels through the structure or characteristics of the unsupervised data itself. After having pseudo-labels, training can be performed like supervised learning, and then valuable representations for downstream tasks can be learned.

[0089] Contrastive learning has achieved remarkable success in the field of self-supervised learning. Contrastive learning learns meaningful representations by comparing semantically consistent and inconsistent sample pairs. Usually, the anchor samples obtained by data augmentation (such as rotation and Gaussian blur in the image) are used as positive samples, and the remaining samples are all used as negative samples, so that the distance between the positive samples and the anchor samples in the representation space is as close as possible, and the distance between the negative samples and the anchor samples in the representation space is as far as possible. However, traditional contrastive learning is too rough in the construction and selection of positive samples. Specifically, the traditional contrastive learning method obtains positive samples by data augmentation of the anchor samples. This positive sample construction method cannot identify other potential positive samples in the training data that are of the same category as the anchor samples. In traditional contrastive learning, these potential positive samples are treated as negative samples, which reduces the quality of the contrastive learning representation, thereby having a negative impact on the effect of downstream emotion recognition tasks.

[0090] It should be noted that the defects existing in the solutions in the above-mentioned prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above-mentioned problems and the solutions proposed in the embodiments of the present invention for the above-mentioned problems below should all be the contributions made by the inventor to the present invention in the process of the present invention.

[0091] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. It should be noted that the embodiments of the present invention and the features in the embodiments can also be combined with each other without conflict.

[0092] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In the description of the present invention, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0093] After the above in-depth analysis, the embodiments of the present invention propose a method for constructing an emotion recognition model, an emotion recognition method and a system, which adopt a self-supervised representation learning method, which is different from the traditional supervised learning method that requires a large amount of labeled data. On the one hand, it ensures the low-cost acquisition of a large amount of training data, and on the other hand, it can fully mine the information carried by the EEG signal itself; at the same time, it improves the traditional contrastive learning method, constructs three positive sample mining strategies, and sets multiple positive samples, which effectively improves the quality of contrastive learning representation, thereby effectively improving the accuracy and effect of emotion recognition.

[0094] See also Figure 1 The method for constructing an emotion recognition model provided by an embodiment of the present invention comprises the following steps:

[0095] Step S1, collecting a number of 62-lead EEG signals within a predetermined time period, wherein the subject generates two or more different types of emotions within the predetermined time period, and recording the start and end points of the generation of the emotions.

[0096] In this step, a 62-lead EEG acquisition device is used to collect EEG signals. The EEG acquisition device has a total of 62 electrodes, each of which is attached to the surface of the human head. Each electrode corresponds to a channel and is distributed in the left and right hemispheres and the front and back areas of the head. The above-mentioned electrodes have no effect on the stimulation of emotions and are carried out under the condition of ensuring the complete safety of the subjects.

[0097] The different emotions are distinguished according to the emotion types that need to be identified. For example, they are distinguished into four emotion types: happiness, sadness, fear, and neutrality. When collecting data, the subjects are emotionally stimulated through video or image data to produce the corresponding type of emotion, and the EEG signals of the emotion under this type are collected. Of course, other emotion types can also be included or more detailed distinctions can be made, such as anger.

[0098] Step S2, denoising and filtering the collected 62-lead EEG signals to retain the effective bands in the EEG signals; and dividing the effective bands according to the start and end points of the time when the emotions are generated, retaining the effective EEG signal segments between the start and end points.

[0099] In this step, the data is denoised, and the collected raw EEG signals are separated into independent components using the independent component analysis method ICA, and various artifacts (such as eye movements, electromyography, electrocardiography, etc.) mixed in the EEG signals are removed. The raw EEG data collected from the scalp can be regarded as the sum of the real EEG signal and various artifacts. In theory, it is believed that pure EEG signals and artifacts are independent of each other. Therefore, the raw data can be decomposed by ICA, the artifacts can be eliminated, and a pure EEG signal can be obtained.

[0100] The formula for independent component analysis when eliminating artifacts is as follows:

[0101] X=WS (1)

[0102] In formula (1), X represents the mixed signal of EEG signal and artifact, that is, the original signal collected directly from the scalp, W represents the ICA unmixing matrix, and S represents various independent signals.

[0103] The above formula can be used to obtain pure EEG signals and various independent artifacts. Different signal frequencies and amplitudes are quite different, so they can be easily distinguished. In order to eliminate the influence of artifacts on the experimental results, after separation, the artifact signals are removed from the original data to obtain pure EEG signals.

[0104] The filtering uses a bandpass filter between 0.5 Hz and 75 Hz to process the pure EEG signal, obtains the effective band between 0.5 Hz and 75 Hz, and filters out the invalid band above 75 Hz and below 0.5 Hz. Each effective EEG signal segment is the EEG signal of the corresponding emotional time point of 62 leads containing the effective band.

[0105] In this step, since the EEG signal is a full record of the predetermined time period, the only useful signals are those when emotions are generated. Therefore, the EEG signal is divided into segments of appropriate length according to the recorded start and end points of the time when the emotions are generated, and the EEG signals outside this range are discarded. Taking the SEED-IV dataset as an example, the subjects are emotionally stimulated through movie clips. A single experiment includes 5 seconds of prompt time, 2 minutes of movie clip time, and 45 seconds of self-assessment time. During the EEG collection process, the start and end points of the time when the subjects watch the movie clips are recorded. Only the emotions generated when watching the movie clips are useful EEG signals.

[0106] Step S3, among all valid EEG signal segment samples, positive samples are screened based on temporal proximity to obtain a first positive sample subset.

[0107] In this step, the screening is based on temporal proximity. Since time series have local smoothness, time series data should evolve slowly over time. For example, in the context of emotion recognition, each different emotion lasts for a period of time, and it is difficult for emotions to change suddenly in a very short time. Therefore, it can be assumed that the samples that are temporally close to the anchor sample have the same label as the anchor sample. The specific process is as follows:

[0108] Step S31, randomly select an anchor sample x a ; Define a hyperparameter T to represent the size of the constructed time window. In the anchor sample x a Define a time window of length 2T around in Represents the anchor sample x a The corresponding time.

[0109] Step S32: based on all samples x except the anchor sample i The sample selection constraints are constructed as follows:

[0110]

[0111] and:

[0112]

[0113] In formula (2), is the sample x i Is it in the time window? Determined by i is the sample label value, and the value 1 represents the constraint condition Q i Sample x in i is a positive sample, and the value is -1, which represents the constraint condition Q i Sample x in i is a negative sample.

[0114] Step S4, in each valid EEG signal segment, extract the five bands of Delta wave (0.5-4Hz), Theta wave (4-8Hz), Alpha wave (8-13Hz), Beta wave (13-32Hz) and Gamma wave (>32Hz) contained in each lead EEG signal; extract the power spectral density of the five bands respectively, and calculate the differential entropy features of the five bands respectively according to the power spectral density, and then construct the node feature representation of the current segment according to the differential entropy features of the five bands.

[0115] In this step, the average power spectral density PSD of each of the five bands of each pilot signal can be expressed as:

[0116] PSD={∑[abs(FFTDATA i )]} / (EndNum-StartNum+1) (3)

[0117] In formula (3), FFTDATA i Refers to the signal value corresponding to the i-th time point in the valid EEG signal segment. EndNum is the end time point of the valid EEG signal segment, and StartNum is the start time point of the valid EEG signal segment.

[0118] Calculate the power spectrum density corresponding to the five bands of each lead signal, and approximate the differential entropy feature DE of the corresponding band under the current lead based on the power spectrum density:

[0119] DE=log2(PSD) (4)

[0120] The differential entropy eigenvalues ​​corresponding to the five bands of each guide signal are combined into a matrix of size 62×5 as the node feature representation of the current valid EEG signal segment:

[0121]

[0122] In formula (5), the matrix H is the node feature representation of the valid EEG signal segment.

[0123] Step S5, among all valid EEG signal segment samples, positive samples are screened based on the similarity of node feature representation to obtain a second positive sample subset.

[0124] In this step, the positive samples are screened based on the similarity of the node feature representation. The specific process is as follows:

[0125] Step S51, calculate the differential entropy matrix H of the anchor samples respectively a And the differential entropy matrix H of the remaining samples i , i∈{1,2,...,batch size-1}, calculate the Mahalanobis distance between two matrices:

[0126]

[0127] In formula (6), Represents the anchor sample differential entropy matrix H a The value of the element in row j and column k, Represents the differential entropy matrix H of the i-th sample i The value of the element in the jth row and the kth column, Σ -1 is the inverse matrix of the covariance matrix of the two differential entropy matrices.

[0128] The most common method for calculating the distance between two points is the Euclidean distance, but the Euclidean distance also has obvious shortcomings. It treats the differences between different attributes of samples as equal, which sometimes cannot meet practical requirements. The Mahalanobis distance is an effective method for calculating the similarity between two unknown samples. Unlike the Euclidean distance, it can take into account the relationship between various features. In terms of emotion recognition tasks, calculating the Mahalanobis distance between two samples can comprehensively consider the correlation between 62 leads and 5 frequency bands, which is better than the Euclidean distance.

[0129] Step S52, after calculating the Mahalanobis distance, the obtained (batch size-1) Mahalanobis distances are arranged in order from small to large, the proportional coefficient k1 is set, the first (batch size×k1) are selected as positive samples to form the second positive sample subset, and the remaining samples are used as negative samples.

[0130] The differential entropy is applied to the screening of positive samples because different emotions have different energies in different frequency bands, while the same emotion has similar characteristic energies in a specific frequency band. Using the similarity of differential entropy features to measure the similarity of node feature representations can fully mine potential samples with the same labels as anchor samples as positive samples, thereby improving the quality of contrastive learning representation.

[0131] Step S6, constructing a brain network graph G based on the valid EEG signal segments: G = (V, E); where V = {V1, V2, ..., V 62} is a set of vertices corresponding to the 62 leads; E is an edge connecting the vertices, and whether an edge between two vertices exists is determined according to the phase-locked value; the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment is calculated.

[0132] In this step, the phase-locking value between vertices is used to measure the connectivity between leads, that is, the existence of edges between vertices, and its formula is expressed as:

[0133]

[0134] In formula (7), is the phase difference between the two lead signals i and j, t represents time, ω represents angular frequency, and N is the number of sampling points in the time range from the start time point StartNum to the end time point EndNum of the valid EEG signal segment.

[0135] The phase-locking value between the leads is compared with the preset threshold δ. If the phase-locking value is greater than or equal to the threshold δ, it is determined that there is a connection relationship between the brain areas corresponding to the two leads, and there is an edge E between the two vertices; if the phase-locking value is less than the threshold δ, it is determined that there is no connection relationship between the brain areas corresponding to the two leads, and there is no edge E between the two vertices.

[0136] For the brain network graph G, define the adjacency matrix of the graph as A. For the element a in A ij , calculated as follows:

[0137]

[0138] For the brain network graph G, its adjacency matrix is ​​a 62×62 symmetric matrix, expressed as follows:

[0139]

[0140] In formula (9), matrix A is the adjacency matrix corresponding to the brain network graph G corresponding to the valid EEG signal segment.

[0141] Step S7, constructing a topological structure feature representation of each brain network graph G according to the adjacency matrix A.

[0142] In this step, the topological structure feature representation includes three factors, namely, average path length, clustering coefficient, and connection rate.

[0143] Among them, the average path length is defined to characterize the length of the communication link between nodes as a whole:

[0144]

[0145] In formula (10), N V Indicates the number of nodes in the network, preferably N in this embodiment V =62. ij It represents the length of the shortest path among all paths connecting nodes (i, j).

[0146] The clustering coefficient is defined as the ratio of the number of connections between the nearest neighbors of a node to the maximum possible number of connections:

[0147]

[0148] and:

[0149]

[0150] In formula (11), N V Indicates the number of nodes in the network, preferably N in this embodiment V =62. k i Indicates the number of nodes directly connected to node i in the network, N Ei These k i The actual number of edges between nodes.

[0151] Define the connection rate to measure N V The strength of the connection between nodes:

[0152]

[0153] In formula (12), N V Indicates the number of nodes in the network, preferably N in this embodiment V =62. N E Represents the number of edges in the network.

[0154] The topological eigenvector λ is obtained by equations (10)-(12) = (L glob (G), C(G), CR(G)), where the vector λ represents the topological structure feature of the effective EEG signal segment.

[0155] Step S8, using valid EEG signal segments as samples, and screening positive samples based on the similarity of the topological structure feature representation of the brain network graph G to obtain a third positive sample subset.

[0156] The positive sample screening is performed based on the similarity of the topological structure of the brain network graph G, that is, the similarity of the topological structure feature representation. The specific process is as follows:

[0157] Step S81, randomly selected anchor sample x a , respectively find the anchor sample x a Brain network graph Ga The corresponding adjacency matrix A a And the remaining samples x i Brain network graph G i The corresponding adjacency matrix A i ,i∈{1,2,...,batch size-1}.

[0158] The anchor sample x is calculated by equations (10)-(12) respectively. a The topological structure characteristics of a =(L glob (G a ),C(G a ),CR(G a )) and the remaining samples x i The topological structure characteristics of i =(L glob (G i ),C(G i ),CR(G i )).

[0159] Compute the cosine distance between two topological feature representations:

[0160]

[0161] In formula (13), represents the value of the jth element of the topological structure feature representation of the anchor sample, Represents the value of the jth element represented by the topological structure feature of the i-th sample.

[0162] Step S82, after calculating the cosine distance, arrange the obtained (batch size-1) cosine distances in order from small to large, set the proportional coefficient k2, select the first (batch size×k2) as positive samples to form the third positive sample subset, and the remaining samples as negative samples.

[0163] The topological structure feature representation is applied to the screening of positive samples because the functional connections of the brain under different emotions are quite different, while the functional connections of the brain under the same emotion are similar. The similarity of the functional topological structure of the brain network is measured by the similarity of the topological structure representation, such as Figure 2 As shown in the figure, taking the convolutional neural network CNN as an example, after collecting 62-lead EEG signals, the brain network graph G is constructed with the CNN topology structure, and the anchor sample is selected to represent z a , output positive sample representation z p , negative sample representation z n , where the subscript represents the sample number. After calculation, it can fully mine potential samples with the same label as the anchor sample as positive samples, improving the quality of contrastive learning representation.

[0164] After completing the three positive sample screenings, the positive samples screened out by the three positive sample screening strategies are combined to obtain the final positive sample set, and the remaining samples are all negative samples.

[0165] Step S9, taking the union of the first positive sample subset, the second positive sample subset and the third positive sample subset to obtain a final positive sample set, and the remaining samples are negative samples; the convolutional neural network is trained by the final positive sample set and negative sample set; a classification layer is added to the pre-trained convolutional neural network, a small amount of labeled sample data is randomly selected, and the classification layer is trained so that the parameters of the classification layer are adapted to the emotion recognition task, and an emotion recognition model is obtained.

[0166] In this step, the convolutional neural network is trained by obtaining the final positive sample set and negative sample set, and the output of the convolutional neural network is used as the representation of the positive and negative samples. i , i∈{1,2,...,batch size-1}, and substitute it into the loss function for calculation; when the loss function value is less than the set threshold, the convolutional neural network training is completed, which is equivalent to the pre-training process of the recognition model.

[0167] In this step, the loss function is expressed as:

[0168]

[0169] In formula (14), s i,j For z i and z j The cosine similarity s i,j =z i T z j / ‖z i ‖‖z i ‖; subscript i represents anchor samples, subscript p represents positive samples, s i,n For z i and z n The cosine similarity s i,n =z i T z n / ‖z i ‖‖z i ‖, subscript n represents negative samples; P(i) represents the set of all positive samples, N(i) represents the set of all negative samples; τ p represents the temperature coefficient of the positive sample, τ n Represents the temperature coefficient of the negative samples.

[0170] Compared with the traditional contrastive learning InfoNCE loss that considers the relationship between one positive sample and multiple negative samples, the loss function shown in formula (14) expands the contrastive loss to multiple positive samples and multiple negative samples. This change ensures that the model can cluster samples belonging to the same class more closely. Samples belonging to the same class will be regarded as positive samples to each other, ensuring the learning of intra-class features and effectively improving the performance of contrastive learning.

[0171] After finding the comparative loss function, the model is optimized through the back propagation algorithm of the neural network in order to obtain a lower loss function value and iterate repeatedly; when the loss value is less than the set threshold, the optimal network model is obtained.

[0172] In order to make the obtained network model better adapt to the downstream emotion recognition task, a classification layer is added to the pre-trained convolutional neural network, and a small amount of labeled data in step S2 is randomly selected to train the classification layer so that the parameters of the classification layer are adapted to the emotion recognition task, thereby obtaining an emotion recognition model.

[0173] It can be seen from the above technical solutions that the method for constructing the emotion recognition model provided in this embodiment is based on a self-supervised representation learning method. For the collected EEG signals representing emotions, two positive sample mining strategies are used to obtain positive and negative sample sets. This not only ensures the low-cost acquisition of a large amount of training data, but also improves the quality of contrastive learning representation. It can fully mine the information carried by the EEG signal itself, thereby effectively improving the accuracy and precision of the model.

[0174] Based on the emotion recognition model constructed by the above-mentioned emotion recognition model construction method, an embodiment of the present invention further provides an emotion recognition method, the method comprising:

[0175] Obtain valid EEG signal segments corresponding to the emotion to be identified. The valid EEG signal segments corresponding to the emotion to be identified here first collect the EEG signals of at least one lead in the time period corresponding to the emotion, denoise the EEG signals to obtain pure EEG signals, and then perform filtering to obtain valid EEG signal segments; at the same time, extract the five bands of Delta wave (0.5-4Hz), Theta wave (4-8Hz), Alpha wave (8-13Hz), Beta wave (13-32Hz) and Gamma wave (>32Hz) from the valid EEG signal segments.

[0176] The effective EEG signal fragment is input into a mature emotion recognition model, and the recognized emotion category is output.

[0177] Based on the same idea, an embodiment of the present invention also provides an emotion recognition system, which includes: a 62-lead EEG signal acquisition module, a sample extraction module, a first positive sample screening module, a second positive sample screening module, a third positive sample screening module, a sample set synthesis module, a recognition model training module, and a recognition result output module.

[0178] The 62-lead EEG signal acquisition module is used to collect several 62-lead EEG signals within a predetermined time period, in which the subject generates two or more different types of emotions, and records the start and end points of the emotions as basic data of the sample; and is also used to collect at least one lead EEG signal corresponding to the emotion to be identified;

[0179] The sample extraction module is used to filter the collected 62-lead EEG signals after denoising, retaining the effective bands in the EEG signals; and to divide the effective bands according to the start and end points of the time when the emotions are generated, retaining the effective EEG signal segments between the start and end points;

[0180] The first positive sample screening module is used to screen positive samples based on temporal proximity among all valid EEG signal segment samples to obtain a first positive sample subset;

[0181] The second positive sample screening module is used to extract the five bands of Delta wave (0.5-4Hz), Theta wave (4-8Hz), Alpha wave (8-13Hz), Beta wave (13-32Hz) and Gamma wave (>32Hz) contained in each lead EEG signal in each valid EEG signal segment; extract the power spectrum density of the five bands respectively, and calculate the differential entropy features of the five bands respectively according to the power spectrum density, and then construct the node feature representation of the current segment according to the differential entropy features of the five bands; and also used to screen positive samples based on the similarity of the node feature representation to obtain the second positive sample subset;

[0182] The third positive sample screening module is used to construct a brain network graph G based on valid EEG signal segments: G = (V, E); wherein V = {V1, V2, ..., V 62} is a set of vertices corresponding to the 62 leads; E is an edge connecting the vertices, and the existence of an edge between two vertices is determined according to the phase-locked value, and the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment is calculated, and the topological structure feature representation of each brain network graph is constructed according to the adjacency matrix A; it is also used to screen positive samples based on the similarity of the topological structure feature representation of the brain network graph G to obtain a third positive sample subset;

[0183] The sample set synthesis module is used to take the union of the first positive sample subset, the second positive sample subset and the third positive sample subset to obtain a final positive sample set, and the remaining samples are negative samples;

[0184] The recognition model training module is used to train the convolutional neural network through the final positive sample set and negative sample set obtained, and use the output of the convolutional neural network as the representation of the positive and negative samples, and substitute it into the loss function for calculation; when the loss function value is less than the set threshold, the training of the convolutional neural network is completed; a classification layer is added to the convolutional neural network obtained by pre-training, a small amount of labeled data is randomly selected, and the classification layer is trained so that the parameters of the classification layer are adapted to the emotion recognition task, and a mature emotion recognition model is obtained;

[0185] The recognition result output module is used to input the valid EEG signal segments corresponding to the emotion to be recognized into the emotion recognition model and output the emotion recognition result.

[0186] In this embodiment, each module is implemented by a processor, and a memory is appropriately added when storage is required. Among them, the processor can be but is not limited to a microprocessor MPU, a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0187] It should be noted that the emotion recognition system described in this embodiment corresponds to the emotion recognition method and the resulting model building method. The description and limitation of the emotion recognition method and the model building method are also applicable to the emotion recognition system in this embodiment and will not be repeated here.

[0188] A person skilled in the art can understand that all or part of the processes and modules in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0189] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for constructing an emotion recognition model, characterized in that: The steps include: Step S1, collecting a number of 62-lead EEG signals within a predetermined time period, wherein the subject generates two or more different types of emotions within the predetermined time period, and recording the start and end points of the generation of the emotions; Step S2, filtering the collected 62-lead EEG signals after denoising, retaining the effective bands in the EEG signals; and dividing the effective bands according to the start and end points of the time when the emotions are generated, retaining the effective EEG signal segments between the start and end points; Step S3, among all valid EEG signal segment samples, positive samples are screened based on temporal proximity to obtain a first positive sample subset; Step S4, in each valid EEG signal segment, extract the five bands of Delta wave, Theta wave, Alpha wave, Beta wave and Gamma wave contained in each lead EEG signal; extract the power spectrum density of the five bands respectively, and calculate the differential entropy features of the five bands respectively according to the power spectrum density, and then construct the node feature representation of the current segment according to the differential entropy features of the five bands; Step S5, among all valid EEG signal segment samples, positive samples are screened based on the similarity of node feature representation to obtain a second positive sample subset; Step S6, constructing a brain network graph G based on the valid EEG signal segments: G = (V, E); Where V = {V1, V2, ..., V 62 } is a set of vertices corresponding to the 62 leads; E is an edge connecting the vertices, and whether an edge between two vertices exists is determined according to the phase-locked value; the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment is calculated; Among them, the phase lock value PLV ij It is used to measure the connectivity between leads. The calculation formula is: In formula (7), is the phase difference between the two lead signals i and j, t represents time, ω represents angular frequency, and N is the number of sampling points in the time range from the start time point StartNum to the end time point EndNum of the valid EEG signal segment; Compare the phase-locking value between the leads with the preset threshold δ. If the phase-locking value is greater than or equal to the threshold δ, it is determined that there is a connection relationship between the brain regions corresponding to the two leads, and there is an edge E between the two vertices. If the phase-locking value is less than the threshold δ, it is determined that there is no connection relationship between the brain regions corresponding to the two leads, and there is no edge E between the two vertices. Step S7, constructing a topological structure feature representation of each brain network graph G according to the adjacency matrix A; Step S8, using valid EEG signal segments as samples, performing positive sample screening based on the similarity of the topological structure feature representation of the brain network graph G, and obtaining a third positive sample subset; Step S9, taking the union of the first positive sample subset, the second positive sample subset and the third positive sample subset to obtain a final positive sample set, and the remaining samples are negative samples; the convolutional neural network is trained by the final positive sample set and negative sample set; a classification layer is added to the pre-trained convolutional neural network, a small amount of labeled sample data is randomly selected, and the classification layer is trained so that the parameters of the classification layer are adapted to the emotion recognition task, and an emotion recognition model is obtained.

2. The method for constructing an emotion recognition model according to claim 1, characterized in that: In step S2, the denoising is performed by using the independent component analysis method ICA to separate the independent components in the EEG signal and remove various artifacts mixed in the EEG signal; The formula for independent component analysis ICA when eliminating artifacts is as follows: X=WS (1) In formula (1), X represents the mixed signal of EEG signal and artifact, that is, the original signal collected directly from the scalp, W represents the ICA unmixing matrix, and S represents various independent signals.

3. The method for constructing an emotion recognition model according to claim 1, characterized in that: The specific process of step S3 is as follows: Step S31, randomly select an anchor sample x a ; Define a hyperparameter T to represent the size of the constructed time window; in the anchor sample x a Define a time window of length 2T around in Represents the anchor sample x a The corresponding time; Step S32: based on all samples x except the anchor sample i The sample selection constraints are constructed as follows: and: In formula (2), is the sample x i Is it in the time window? Determined by i is the sample label value, and the value 1 represents the constraint condition Q i Sample x in i is a positive sample, and the value is -1, which represents the constraint condition Q i Sample x in i is a negative sample.

4. The method for constructing an emotion recognition model according to claim 1, characterized in that: The average power spectral density PSD of each of the five bands of each pilot signal in step S4 is expressed as: PSD={∑[abs(FFTDATA i )]} / (EndNum-StartNum+1) (3) In formula (3), FFTDATA i It refers to the signal value corresponding to the i-th time point in the valid EEG signal segment; EndNum is the end time point of the valid EEG signal segment, and StartNum is the start time point of the valid EEG signal segment; Calculate the power spectrum density corresponding to the five bands of each lead signal, and approximate the differential entropy feature DE of the corresponding band under the current lead based on the power spectrum density: DE =log2(PSD) (4) The differential entropy eigenvalues ​​corresponding to the five bands of each guide signal are combined into a matrix of size 62×5 as the node feature representation of the current valid EEG signal segment: In formula (5), the matrix H is the node feature representation of the valid EEG signal segment.

5. The method for constructing an emotion recognition model according to claim 4, characterized in that: The positive sample screening is performed based on the similarity of node feature representation, and the specific process is as follows: Step S51, respectively calculate the differential entropy matrix H of the anchor samples a And the differential entropy matrix H of the remaining samples i , i∈{1,2,...,batch size-1}, calculate the Mahalanobis distance between two matrices: In formula (6), Represents the anchor sample differential entropy matrix H a The value of the element in row j and column k, Represents the differential entropy matrix H of the i-th sample i The value of the element in the jth row and the kth column, Σ -1 is the inverse matrix of the covariance matrix of the two differential entropy matrices; Step S52, after calculating the Mahalanobis distance, the obtained (batch size-1) Mahalanobis distances are arranged in order from small to large, the proportional coefficient k1 is set, the first (batch size×k1) are selected as positive samples to form the second positive sample subset, and the remaining samples are used as negative samples.

6. The method for constructing an emotion recognition model according to claim 1, characterized in that: The calculation of the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment specifically includes: For the brain network graph G, define the adjacency matrix of the graph as A. For the element a in A ij , calculated as follows: For the brain network graph G, its adjacency matrix is ​​a 62×62 symmetric matrix, expressed as follows: In formula (9), matrix A is the adjacency matrix corresponding to the brain network graph G corresponding to the valid EEG signal segment.

7. The method for constructing an emotion recognition model according to claim 6, characterized in that: The topological structure feature representation includes three factors, namely, average path length, clustering coefficient, and connection rate; among them, The average path length is defined as formula (10), which is used to characterize the length of the communication link between nodes as a whole: In formula (10), N V represents the number of nodes in the network, d ij Represents the length of the shortest path among all paths connecting nodes (i, j); The clustering coefficient is defined as formula (11), which is used to represent the ratio of the number of connections between the nearest neighbors of a node to the maximum possible number of connections: and: In formula (11), N V represents the number of nodes in the network, k i Indicates the number of nodes directly connected to node i in the network, N Ei These k i The number of edges that actually exist between nodes; The connection rate is defined as formula (12), which is used to measure N V The strength of the connection between nodes: In formula (12), N V Represents the number of nodes in the network, N E Represents the number of edges in the network; The topological structure characteristic represents λ=(L glob (G), C (G), CR (G)).

8. The method for constructing an emotion recognition model according to claim 7, characterized in that: The positive sample screening is performed based on the similarity of the topological structure feature representation of the brain network graph G. The specific process is as follows: Step S81, randomly selected anchor sample x a , respectively find the anchor sample x a Brain network graph G a The corresponding adjacency matrix A a And the remaining samples x i Brain network graph G i The corresponding adjacency matrix A i ,i∈{1,2,...,batch size-1}; The anchor sample x is calculated by equations (10)-(12) respectively. a The topological structure characteristics of a =(L glob (G a ),C(G a ),CR(G a )) and the remaining samples x i The topological structure characteristics of i =(L glob (G i ),C(G i ),CR(G i )); Compute the cosine distance between two topological feature representations: In formula (13), represents the value of the jth element of the topological structure feature representation of the anchor sample, Represents the value of the jth element represented by the topological structure feature of the i-th sample; Step S82, after calculating the cosine distance, arrange the obtained (batch size-1) cosine distances in order from small to large, set the proportional coefficient k2, select the first (batch size×k2) as positive samples to form the third positive sample subset, and the remaining samples as negative samples.

9. An emotion recognition method, characterized in that: The method comprises: Obtaining valid EEG signal segments corresponding to the emotion to be identified; firstly collecting the EEG signal of at least one lead in the time period corresponding to the emotion, denoising the EEG signal to obtain a pure EEG signal, and then filtering to obtain the valid EEG signal segment; and extracting five wavebands of Delta wave, Theta wave, Alpha wave, Beta wave and Gamma wave from the valid EEG signal segment; The effective EEG signal segment is input into the emotion recognition model constructed according to any one of claims 1 to 8, and the recognized emotion category is output.

10. An emotion recognition system, characterized in that: The system includes: a 62-lead EEG signal acquisition module, a sample extraction module, a first positive sample screening module, a second positive sample screening module, a third positive sample screening module, a sample set synthesis module, a recognition model training module, and a recognition result output module; wherein, The 62-lead EEG signal acquisition module is used to collect a number of 62-lead EEG signals within a predetermined time period, in which the subject generates two or more different types of emotions, and records the start and end points of the emotions as basic data of the sample; and is also used to collect at least one lead EEG signal corresponding to the emotion to be identified; The sample extraction module is used to filter the collected 62-lead EEG signals after denoising, retaining the effective bands in the EEG signals; and to divide the effective bands according to the start and end points of the time when the emotions are generated, retaining the effective EEG signal segments between the start and end points; The first positive sample screening module is used to screen positive samples based on temporal proximity among all valid EEG signal segment samples to obtain a first positive sample subset; The second positive sample screening module is used to extract the five wave bands of Delta wave, Theta wave, Alpha wave, Beta wave and Gamma wave contained in each lead EEG signal in each valid EEG signal segment; extract the power spectrum density of the five wave bands respectively, and calculate the differential entropy features of the five wave bands respectively according to the power spectrum density, and then construct the node feature representation of the current segment according to the differential entropy features of the five wave bands; and is also used to screen positive samples based on the similarity of the node feature representation to obtain a second positive sample subset; The third positive sample screening module is used to construct a brain network graph G based on valid EEG signal segments: G = (V, E); wherein V = {V1, V2, ..., V 62 } is a set of vertices corresponding to the 62 leads; E is an edge connecting the vertices, and the existence of an edge between two vertices is determined according to the phase-locked value, and the adjacency matrix A of the brain network graph constructed by each valid EEG signal segment is calculated, and the topological structure feature representation of each brain network graph is constructed according to the adjacency matrix A; it is also used to screen positive samples based on the similarity of the topological structure feature representation of the brain network graph G to obtain a third positive sample subset; The sample set synthesis module is used to take the union of the first positive sample subset, the second positive sample subset and the third positive sample subset to obtain a final positive sample set, and the remaining samples are negative samples; The recognition model training module is used to train the convolutional neural network through the final positive sample set and negative sample set obtained, and use the output of the convolutional neural network as the representation of the positive and negative samples, and substitute it into the loss function for calculation; when the loss function value is less than the set threshold, the training of the convolutional neural network is completed; a classification layer is added to the convolutional neural network obtained by pre-training, a small amount of labeled data is randomly selected, and the classification layer is trained so that the parameters of the classification layer are adapted to the emotion recognition task, and a mature emotion recognition model is obtained; The recognition result output module is used to input the valid EEG signal segments corresponding to the emotion to be recognized into the emotion recognition model and output the emotion recognition result.

Images