Method and system for detecting interpersonal nerve synchronization under audio-visual stimulation

Abstract

The invention discloses a method for detecting interpersonal nerve synchronization under audio-visual stimulation, which comprises the following steps of: designing a double-person super-scanning experiment normal form, collecting double-person electroencephalogram signals, and constructing a database of visual stimulation and auditory stimulation; preprocessing the data in the database to obtain electroencephalogram signals of four different frequency bands delta, theta, alpha and beta, extracting the electroencephalogram signals of the alpha frequency band, segmenting the electroencephalogram signals, and calculating a correlation ISC value between subjects of each segment; an intra-brain network and an inter-brain network are constructed by constructing a functional connection matrix, and the similarity of the intra-brain network and the global efficiency of the inter-brain network are calculated, so that the cooperation and synchronization degree between neural activities of subjects under visual stimulation and auditory stimulation is effectively evaluated. The invention further discloses a system for detecting interpersonal nerve synchronization under audiovisual stimulation. According to the method, the difference of subjects is reduced by standardizing experimental conditions, and the influence of single sensory stimulation on an intracerebral network activation mode is studied.

Description

Technical Field

[0001] The present invention relates to the field of multi-person brain-computer interface and electroencephalogram (EEG) ultrascanning technology, and in particular to a method and system for detecting interpersonal neural synchronization under visual and audio stimulation. Background Art

[0002] In the fast-paced world of modern life, the quality of interpersonal interactions profoundly impacts mental health and life experiences. Understanding the underlying neural mechanisms, particularly interpersonal neural synchronization (the synchronization of brain activity across individuals during social interactions), is crucial for understanding social behavior. Traditional neuroscience research primarily focuses on single-brain activity. However, hyperscanning techniques (such as EEG, fMRI, fNIRS, and MEG) reveal the neural synchronization mechanisms underlying interactions by synchronously recording the brain activity of multiple individuals. EEG hyperscanning has become a mainstream method due to its high temporal resolution and low cost.

[0003] Studies have found that interpersonal neural synchronization is widely present in various social scenarios, such as movement coordination, music listening, decision-making cooperation, and face-to-face communication. This synchronization relies on the integration of multisensory (visual, auditory) information and involves higher-order brain areas (such as the cingulate cortex), not just the sensory cortex. In addition, alpha band activity is related to the brain's resting state and attention regulation, and EEG alpha wave synchronization may produce significant oscillatory changes and synchronization in cooperative tasks. Alpha waves correspond to the brain's resting state, and sensory stimulation usually inhibits these oscillations in the corresponding sensory areas of the brain. However, the alpha band is related to a variety of cognitive processes, especially attention processes and social interaction mechanisms.

[0004] Existing research on interpersonal neural synchronization has achieved many results, but still has limitations:

[0005] (1) Existing research is mostly based on natural interactions involving multiple senses (e.g., watching movies, playing musical instruments together), making it difficult to isolate the role of a single sensory channel;

[0006] (2) Although existing studies have only examined the effects of specific sensory channels on interpersonal neural synchronization from the perspective of visual or auditory tasks, there are often differences in subject groups (such as age, cognitive ability, experimental conditions, etc.) between different studies, making it difficult to directly compare the research results.

[0007] Previous studies have mostly compared inter-brain network and inter-subject synchrony, ignoring the synchrony of sensory stimulation on intra-brain network activation patterns. Therefore, a novel method and system for detecting interpersonal neural synchronization under audiovisual stimulation is urgently needed to address these issues. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method and system for detecting interpersonal neural synchronization under audiovisual stimulation, and to study the influence of sensory stimulation on brain network activation patterns by standardizing experimental conditions to reduce subject differences.

[0009] To solve the above technical problems, the present invention adopts a technical solution: providing a method for detecting interpersonal neural synchronization under audiovisual stimulation, comprising the following steps:

[0010] S1: Design a two-person hyperscanning experimental paradigm, collect EEG signals from both individuals, and construct a database of visual and auditory stimuli;

[0011] S2: Preprocessing the data in the database to obtain EEG signals in four different frequency bands, δ, θ, α, and β, extracting the EEG signal in the alpha frequency band and segmenting it to extract the EEG signal under visual and auditory tasks, dividing the EEG signal into several segments according to the time of the stimulus material, calculating the inter-subject correlation (ISC) value of each segment, and analyzing the trend and stability of the subject correlation over time; calculating the ISC value of the entire stimulation process for the EEG signals of two subjects during each task, and using it to evaluate the similarity of brain activity between subjects under the same task or stimulation;

[0012] S3: By constructing a functional connectivity matrix to construct intra-brain networks and inter-brain networks, and calculating the similarity of intra-brain networks and the global efficiency of inter-brain networks, the degree of collaboration and synchronization between neural activities between subjects under visual and auditory stimulation can be effectively evaluated.

[0013] In a preferred embodiment of the present invention, in step S1, the design method of the two-person hyperscanning experimental paradigm includes:

[0014] The experiment consisted of two experimental tasks: a visual task and an auditory task. The visual task used two types of stimulus materials: pictures and videos, while the auditory task used two types of stimulus materials: sentences and music. Each stimulus material was also divided into different types of emotional valence, namely positive, neutral and negative, resulting in a total of 12 different conditions.

[0015] The experiment was conducted in three rounds. The stimulus materials used in each round included pictures, videos, sentences and music. The emotional valence of the stimulus materials in each round was different. In each round of the experiment, the subjects underwent evoked EEG recordings in four natural stimulation sessions, namely watching pictures, listening to sentences with eyes closed, watching videos and listening to music. The videos were silent movie clips and the sentences were non-native sentences.

[0016] In a preferred embodiment of the present invention, in step S2, the step of preprocessing the data in the database includes:

[0017] S201: Preview the EEG signal and remove data with obvious drift;

[0018] S202: Use notch filtering technology to remove 50Hz line noise, and then perform bandpass filtering between 0.1Hz and 45Hz to remove noise that is too high or too low in frequency;

[0019] S203: changing the original reference electrode FCz to an average reference to optimize the signal reference mode;

[0020] S204: Decompose the EEG signal into several independent components through independent component analysis (ICA), and combine it with visual inspection to accurately identify and remove artifacts, thereby effectively eliminating noise components.

[0021] In a preferred embodiment of the present invention, in step S2, the specific steps of calculating the inter-subject correlation (ISC) value of each segment include:

[0022] First, for each pair of subjects k and l, the cross-electrode covariance matrix C is calculated kl , which is:

[0023]

[0024] Among them, X k (t) and X l (t) represent the EEG signals of individuals k and l at time point t, μ k and μ l are the average values of individuals k and l over the entire time series, respectively, and T represents the total number of time points of the signal;

[0025] Then, the following formula was applied to calculate the inter-subject correlation (ISC) value between each pair of subjects:

[0026]

[0027] Among them, V i represents the i-th eigenvector extracted by PCA or other methods, which is used to capture the key features of the signal; T is the total number of time points of the signal.

[0028] In a preferred embodiment of the present invention, in step S3, the specific steps of constructing the intra-brain network and the inter-brain network by constructing the functional connectivity matrix are as follows:

[0029] First, the circular correlation coefficient CCorr is selected as the synchronization indicator:

[0030]

[0031] in, and Represents the phase values of the two channels being compared and the average phase of the two signals, k represents the kth data point, N represents the total number of data points, and Represents the phase value of the kth data point of the two channels;

[0032] Then, the two subjects used a cap equipped with 32 electrodes, and the EEG signals of the two subjects were merged into a two-dimensional array of 64 electrodes. The functional connectivity matrix was constructed using correlation. The nodes in the matrix represent network nodes, and the values represent the weights of the connections between nodes. The first 32 electrodes were the electrodes in the brain of the first subject, and the back 32 electrodes were the electrodes in the brain of the second subject. CCorr was calculated between the electrodes in the brain of the first subject to obtain the brain network of the first subject, CCorr was calculated between the electrodes in the brain of the second subject to obtain the brain network of the second subject, and CCorr was calculated between the electrodes in the brain of the first subject and the electrodes in the brain of the second subject to obtain the inter-brain network.

[0033] In a preferred embodiment of the present invention, in step S3, the similarity of the brain network is calculated using the following formula:

[0034]

[0035] Among them, C 1,k,t represents the edge weight of the first node pair (k, t) in the functional connectivity matrix, C 2,k,t represents the edge weight of the second node pair (k, t) in the functional connection matrix, Ne is the number of nodes; SimiNet∈[0,1], 0 and 1 indicate that the two connection matrices are dissimilar and exactly the same, respectively.

[0036] In a preferred embodiment of the present invention, in step S3, the global efficiency of the inter-brain network is calculated using the following formula:

[0037]

[0038] Where n is the total number of nodes in the network, N is the set of nodes, and d ij is the shortest path length between node i and node j.

[0039] In a preferred embodiment of the present invention, in step S3, under the same stimulation, the higher the similarity of the network structure in the brains of two subjects, the stronger the synchronization between multiple brains; the global efficiency of the cross-brain network is used to measure the efficiency and stability of information transmission between the brain networks of the subjects. The higher the global efficiency, the stronger the ability of the inter-brain network integration and the higher the coordination ability of different electrodes or brain regions, which further indicates high synchronization.

[0040] To solve the above technical problems, another technical solution adopted by the present invention is to provide a system for detecting interpersonal neural synchronization under audiovisual stimulation, comprising:

[0041] The audiovisual stimulation database construction module is used to design a two-person hyperscanning experimental paradigm, collect two-person EEG signals, and build a database of visual and auditory stimulation;

[0042] An EEG data preprocessing module is used to preprocess the data in the database to obtain EEG signals in four different frequency bands: δ, θ, α, and β. The EEG signal in the alpha frequency band is extracted and segmented to extract EEG signals under visual and auditory tasks.

[0043] An inter-subject correlation calculation module is used to divide the EEG signals preprocessed by the EEG data preprocessing module into several segments according to the time of the stimulation material, calculate the inter-subject correlation (ISC) value of each segment, and analyze the trend and stability of the subject correlation over time; by calculating the ISC value of the entire stimulation process for the EEG signals of two subjects during each task, the similarity of brain activity between subjects under the same task or stimulation is evaluated;

[0044] The brain network construction and evaluation module is used to construct intra-brain networks and inter-brain networks by constructing functional connectivity matrices, and calculate the similarity of intra-brain networks and the global efficiency of inter-brain networks, so as to effectively evaluate the degree of collaboration and synchronization between neural activities between subjects under visual and auditory stimulation.

[0045] In a preferred embodiment of the present invention, the step of preprocessing the data in the database by the EEG data preprocessing module includes:

[0046] First, the EEG signals were previewed and data with obvious drift were removed;

[0047] Secondly, notch filtering technology is used to remove 50Hz line noise, and then band-pass filtering is performed between 0.1Hz and 45Hz to remove noise that is too high or too low in frequency;

[0048] Next, the original reference electrode FCz was changed to an average reference to optimize the signal reference method;

[0049] Finally, the EEG signal is decomposed into several independent components through independent component analysis (ICA), and combined with visual inspection, the artifacts are accurately identified and removed, thereby effectively eliminating the noise component.

[0050] The present invention has the following beneficial effects: It focuses on the differences in the roles of single sensory channels—vision and hearing—in interpersonal neural synchronization in the alpha band, standardizes experimental conditions to reduce inter-subject variability, and studies the impact of sensory stimulation on brain network activation patterns. Understanding the impact of different sensory stimuli on interpersonal neural synchronization can help design more natural and intuitive interactive interfaces, improve interaction efficiency and experience, and contribute to a deeper understanding of the impact of interpersonal neural synchronization on social interaction and mental health, providing a scientific basis for improving interpersonal interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 1 is a flow chart of a method for detecting interpersonal neural synchronization under audiovisual stimulation according to the present invention;

[0052] Figure 2 is a schematic diagram of the design of the two-person hyperscanning experimental paradigm;

[0053] Figure 3 is a schematic diagram of the functional connectivity matrix within and between subjects’ brains;

[0054] Figure 4 is a schematic diagram of the correlation between subjects under different conditions;

[0055] Figure 5 This is a schematic diagram of the similarity of brain networks among subjects under different conditions;

[0056] Figure 6 Schematic diagram of the global efficiency of the inter-brain network of subjects under different conditions;

[0057] Figure 7 4 is a structural block diagram of the system for detecting interpersonal neural synchronization under audiovisual stimulation. DETAILED DESCRIPTION

[0058] The preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more precise definition of the protection scope of the present invention.

[0059] See also Figure 1 , embodiments of the present invention include:

[0060] A method for detecting interpersonal neural synchronization under audiovisual stimulation comprises the following steps:

[0061] S1: Design a two-person hyperscanning experimental paradigm, collect EEG signals from both individuals, and construct a database of visual and auditory stimuli;

[0062] The primary objective of this experiment was to investigate the effects of different sensory stimuli on interpersonal neural synchronization. Two experimental tasks were designed: a visual task and an auditory task. In each task, two types of stimulus materials were used: pictures and videos in the visual task, and sentences and music in the auditory task. Furthermore, each stimulus material was categorized into different emotional valences: positive, neutral, and negative, resulting in a total of 12 (2 × 2 × 3) different conditions. The experiment was conducted over three rounds, with the stimulus materials used in each round encompassing pictures, videos, sentences, and music. However, the emotional valence of these materials varied across each round. Figure 2 The detailed experimental procedures are shown. In order to balance the potential influence of audiovisual materials and emotional value on the experimental results, the order of material presentation was changed after five sets of experiments.

[0063] Upon entering the recording room, participants were instructed to close their eyes, relax, and sit quietly for three minutes to collect resting-state EEG as baseline activity. Then, participants entered the task-induced phase. In each experimental session, participants underwent evoked EEG recordings during four naturalistic stimulation sessions: viewing pictures, listening to sentences, watching videos, and listening to music. In the picture-viewing module, participants viewed 20 pictures over a one-minute period, each picture playing for three seconds before automatically advancing to the next. After a 30-second rest, the picture-viewing module switched to the music-listening module, where participants closed their eyes and listened to 20 German sentences over a one-minute period. Each sentence played for three seconds before automatically advancing to the next sentence. After a 30-second rest, the module switched to the movie-clips module, where participants watched a one-minute silent film clip. After a 30-second rest, they listened to one minute of pure German music. It is noteworthy that the German sentences and music were chosen as experimental materials to control for interference from semantic content and individual cognition on the measurement of emotional dimensions. All subjects were native Chinese speakers and had no knowledge of German. This ensured that any emotional reactions were primarily due to the sound and valence, rather than the content, of the language. During visual stimulation, subjects were instructed to focus on the computer screen. During auditory stimulation, subjects were instructed to close their eyes and quietly listen to the speech or music. Throughout the experiment, subjects were instructed to remain silent and minimize body movements. There was no eye contact, physical contact, or any other interaction between subjects.

[0064] Among them, the rest time in the above experimental paradigm can be set arbitrarily.

[0065] S2: Preprocessing the data in the database to obtain EEG signals in four different frequency bands, δ, θ, α, and β; extracting the EEG signal in the alpha frequency band and segmenting it to extract the EEG signal under visual and auditory tasks; dividing the EEG signal into several segments according to the time of the stimulus material; calculating the intersubject correlation (ISC) value of each segment; and analyzing the trend and stability of the subject correlation over time; calculating the ISC value of the entire stimulation process for the EEG signals of two subjects during each task to evaluate the similarity of brain activity between subjects under the same task or stimulus;

[0066] Figure 1 Part A in the figure shows data acquisition and preprocessing. In experiments, electroencephalogram (EEG) signals are susceptible to various interferences (such as electromyography, electrooculography, electrode failure, and 50 Hz power frequency), which can cause artifacts and affect data analysis accuracy. Therefore, preprocessing the EEG signals to remove artifacts is crucial. The specific steps are as follows: During EEG signal preprocessing, the signals are first previewed to remove data with significant drift. Notch filtering is then used to remove 50 Hz line noise, followed by bandpass filtering between 0.1 Hz and 45 Hz to remove high- and low-frequency noise. Finally, the original reference electrode FCz is replaced with an average reference to optimize the signal referencing. Independent component analysis (ICA) is used to decompose the EEG signals into multiple components, and visual inspection is used to accurately identify and remove artifacts, effectively eliminating noise.

[0067] This example uses ISC to assess the synchronization of brain activity when subjects are exposed to the same sensory stimulation. This method is particularly suitable for naturalistic stimulation paradigms and can measure the consistency of neural responses between different individuals. First, for each pair of subjects k and l, the cross-electrode covariance matrix C is calculated. kl , which is:

[0068]

[0069] Here, X k (t) and X l (t) represents the EEG signals of individuals k and l at time point t. k and μ l are the average values of individual k and l over the entire time series, which is usually obtained by calculating the average value of each individual signal at all moments. T represents the total number of time points of the signal. For each time point t, calculate the signal X of individual k and l. k (t) and X l (t) and their respective average values μ k and μ lThen, the two deviations are summed to measure the degree of synchronization of the changes of the two individual signals at each time point. Then, the product of all time points is averaged to obtain the covariance C kl Finally, the following formula was applied to calculate the inter-subject correlation (ISC) value between each pair of subjects:

[0070]

[0071] Here, v i represents the i-th eigenvector extracted by PCA or other methods, which is used to capture the key features of the signal. T is the total number of time points of the signal. For each pair of time points t, the dot product X between the individual signal and the eigenvector is calculated. k (t)*v i and X l (t)*v i , and sum their squares (numerator and denominator) to obtain the weighted signal strength. The numerator is the sum of the products of the two individual weighted signals, reflecting the signal synchronization; the denominator is the normalized product of the weighted square sum for standardization. The ISC value is the result of calculating these two ratios, indicating the specific eigenvector v i The present invention further investigates the inter-subject ISC in the alpha band. At the scalp level, the EEG signal is divided into ten segments, each lasting 6 seconds. The ISC value of each segment is calculated, and the temporal trend and stability of the correlation between subjects are analyzed.

[0072] The ISC of the entire stimulation period was calculated from the EEG signals of the two subjects during each task (visual task and auditory task) to evaluate the similarity of brain activity between subjects under the same task or stimulation.

[0073] S3: By constructing a functional connectivity matrix to construct intra-brain networks and inter-brain networks, and calculating the similarity of intra-brain networks and the global efficiency of inter-brain networks, the degree of collaboration and synchronization between multiple brains under visual and auditory stimulation can be effectively evaluated.

[0074] In this example, CCorr (circular correlation coefficient) is selected as the synchronization indicator:

[0075]

[0076] CCorr measures the circular covariance of the difference between the observed phase and the expected or mean phase. and Represents the phase values of the two channels being compared and the average phase of the two signals, k represents the kth data point, N represents the total number of data points, and Represents the phase value of the kth data point between the two channels. Essentially, CCorr is the covariance of the phase variances, indicating whether the phases of the two oscillators vary in a similar manner. Positive values (0 < CCorr ≤ 1) indicate positive synchronization between the signals; the closer the value is to 1, the stronger the positive linear correlation. Negative values (-1 ≤ CCorr < 0) indicate negative synchronization between the signals; the closer the value is to -1, the stronger the negative linear correlation. A zero value (CCorr ≈ 0) indicates no linear synchronization between the signals.

[0077] Specifically, the collected EEG is filtered to obtain four different frequency bands (δ, θ, α, β), and CCorr is calculated based on the alpha band and the formula introduced above. Since a cap equipped with 32 electrodes was used, the EEG signals of the two subjects were merged into a two-dimensional array of 64 electrodes (the vertical axis is the electrode, and the horizontal axis is the sampling point). CCorr was used to construct a functional connectivity matrix. The nodes in the matrix represent network nodes, and the numerical values represent the weights of the connections between nodes. By calculating the correlation between electrodes, a 64*64 symmetric matrix was generated. The first 32 electrodes were the electrodes in the brain of the first subject, and the last 32 electrodes were the electrodes in the brain of the second subject. CCorr was calculated between the electrodes in the brain of the first subject to obtain the brain network of the first subject, CCorr was calculated between the electrodes in the brain of the second subject to obtain the brain network of the second subject, and CCorr was calculated between the electrodes in the brain of the first subject and the electrodes in the brain of the second subject to obtain the inter-brain network. As shown Figure 3 Demonstrate functional connectivity between electrodes within and across subjects. Figure 3 a and Figure 3 The d in the figure presents the brain networks of the two subjects respectively, which are presented as symmetrical matrices, clearly showing the CCorr between the electrodes in each subject's brain. Figure 3 b and Figure 3 The c in the figure shows the cross-brain network. Figure 3 The b in the figure shows the strength of brain connections in a certain direction. The CCorr indicator is not directional, so Figure 3 The two figures together depict the synchronization and mutual influence patterns of brain activity between the two subjects during their interaction.

[0078] SimiNet is a method for quantifying the similarity between brain networks. It not only considers the edge weights of the network, but also pays attention to the differences in the spatial positions of network nodes and weighs the corresponding costs of their differences.

[0079]

[0080] Among them, C 1,k,t represents the edge weight of the first node pair (k, t) in the functional connectivity matrix, C 2,k,t represents the edge weight of the second node pair (k, t) in the functional connection matrix, Ne is the number of nodes; SimiNet∈[0,1], 0 and 1 indicate that the two connection matrices are dissimilar and exactly the same, respectively.

[0081] Cross-brain functional connectivity can effectively assess the degree of collaboration and synchronization between multiple brains. Global efficiency quantifies the overall efficiency of information transmission in the entire network. The average inverse shortest path length is called global efficiency. A higher global efficiency of the cross-brain network means that information can be transmitted and integrated in parallel more quickly and efficiently between brain networks. In the context of social interaction, this means that the brains of two people can better coordinate and synchronize when processing information, thereby promoting more efficient communication and emotional exchange. The efficiency and stability of information transmission between the subjects' brain networks are measured by calculating the global efficiency of the cross-brain network.

[0082]

[0083] Where n is the total number of nodes in the network and N is the set of nodes. ij is the shortest path length between node i and node j.

[0084] Figure 1 B in Figure 3 indicates the calculation of inter-subject correlations, including ISC changes and ISC means when the data were divided into ten segments. Figure 1 The C in the equation represents the construction of intra- and inter-brain networks, and the calculation of intra- and inter-brain network similarity and global inter-brain network efficiency. Under the same stimulus, the greater the structural similarity between the two subjects' intra-brain networks, the greater the synchronization. Global efficiency reflects the efficiency of information transmission within the network. Higher global efficiency indicates stronger inter-brain network integration and greater coordination between different electrodes or brain regions, indicating higher synchronization. The results show that both visual and auditory stimulation significantly enhance inter-subject synchronization, with the effect being even greater under visual stimulation.

[0085] The effect of the method of the present invention is described below in conjunction with experimental results:

[0086] Figure 4The experimental results of inter-subject correlations (ISCs) under different conditions show that both visual and auditory stimulation significantly increased ISCs, with visual stimulation causing significantly higher ISCs than auditory stimulation (p<0.001). Further analysis of the mean squared deviation of ISCs across these time periods revealed that the mean squared deviation of visual ISCs was significantly lower than that of auditory ISCs. This indicates that within the alpha band, visual stimulation not only induces stronger neural synchronization but also exhibits greater temporal stability and continuity, with relatively smaller fluctuations. In terms of mean ISC values, visual and auditory stimulation produced significantly higher ISCs compared to the resting state, and there was also a significant difference between the ISC values induced by visual and auditory stimulation (p<0.001).

[0087] Figure 5 The results of inter-subject correlation (ISC) under different task conditions based on valence are shown. Under visual stimulation, the SimiNet similarity of the brain network between the two subjects was significantly higher than that under auditory stimulation (p<0.01), and both were significantly higher than those in the resting state (p<0.01 between the resting state and the visual stimulation condition, and p<0.05 between the resting state and the auditory stimulation condition). These results suggest that visual stimulation significantly enhanced the structural similarity of brain networks between subjects, which may be closely related to the enhanced functional synchronization between brain regions involved in visual processing. Therefore, it can be inferred that under visual stimulation, the brain network structure of the subjects showed higher similarity. In addition, visual and auditory stimulation significantly enhanced the synchronization between the subjects' brain regions, providing important insights into the effects of different sensory modalities on brain network function.

[0088] Figure 6 The results show the network properties of inter-brain networks across subjects under different conditions. The global efficiency of the inter-brain network was found to be significantly greater under visual stimulation than under auditory stimulation, and both were greater than the global efficiency under resting conditions. This finding was confirmed by an independent samples t-test (p < 0.05), indicating that the global efficiency of the inter-brain network differed significantly between subjects under different sensory stimulations.

[0089] See Figure 7 , the example of the present invention also provides a system for detecting interpersonal neural synchronization under audiovisual stimulation, including an audiovisual stimulation database construction module, an EEG data preprocessing module, an inter-subject correlation calculation module, and a brain network construction and evaluation module.

[0090] The audiovisual stimulation database construction module is used to design a two-person hyperscanning experimental paradigm and collect two-person EEG signals to construct a database of visual stimulation and auditory stimulation;

[0091] The EEG data preprocessing module is used to preprocess the data in the database to obtain EEG signals in four different frequency bands δ, θ, α, and β, extract the EEG signal in the alpha frequency band and segment it to extract EEG signals under visual and auditory tasks;

[0092] The inter-subject correlation calculation module is used to divide the EEG signals preprocessed by the EEG data preprocessing module into several segments according to the time of the stimulation material, calculate the inter-subject correlation (ISC) value of each segment, and analyze the trend and stability of the subject correlation over time; by calculating the ISC value of the entire stimulation process for the EEG signals of two subjects during each task, the similarity of brain activity between subjects under the same task or stimulation is evaluated;

[0093] The brain network construction and evaluation module is used to construct intra-brain networks and inter-brain networks by constructing a functional connectivity matrix, and calculate the similarity of intra-brain networks and the global efficiency of inter-brain networks, so as to effectively evaluate the degree of collaboration and synchronization between neural activities between subjects under visual stimulation and auditory stimulation.

[0094] A system for detecting interpersonal neural synchronization under audiovisual stimulation in this example can execute a method for detecting interpersonal neural synchronization under audiovisual stimulation provided by the present invention, can execute any combination of implementation steps of the method example, and has the corresponding functions and beneficial effects of the method.

[0095] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting interpersonal neural synchronization under audiovisual stimulation, characterized in that: The following steps are involved: S1: Design a two-person hyperscanning experimental paradigm, collect EEG signals from both individuals, and construct a database of visual and auditory stimuli; S2: Preprocessing the data in the database to obtain EEG signals in four different frequency bands, δ, θ, α, and β, extracting the EEG signal in the alpha frequency band and segmenting it to extract the EEG signal under visual and auditory tasks, dividing the EEG signal into several segments according to the time of the stimulus material, calculating the inter-subject correlation (ISC) value of each segment, and analyzing the trend and stability of the subject correlation over time; calculating the ISC value of the entire stimulation process for the EEG signals of two subjects during each task, and using it to evaluate the similarity of brain activity between subjects under the same task or stimulation; S3: By constructing a functional connectivity matrix to construct intra-brain networks and inter-brain networks, and calculating the similarity of intra-brain networks and the global efficiency of inter-brain networks, the degree of collaboration and synchronization between neural activities between subjects under visual and auditory stimulation can be effectively evaluated.

2. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S1, the design method of the two-person hyperscanning experimental paradigm includes: The experiment consisted of two experimental tasks: a visual task and an auditory task. The visual task used two types of stimulus materials: pictures and videos, while the auditory task used two types of stimulus materials: sentences and music. Each stimulus material was also divided into different types of emotional valence, namely positive, neutral and negative, resulting in a total of 12 different conditions. The experiment was conducted in three rounds. The stimulus materials used in each round included pictures, videos, sentences and music. The emotional valence of the stimulus materials in each round was different. In each round of the experiment, the subjects underwent evoked EEG recordings in four natural stimulation sessions, namely watching pictures, listening to sentences with eyes closed, watching videos and listening to music. The videos were silent movie clips and the sentences were non-native sentences.

3. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S2, the step of preprocessing the data in the database includes: S201: Preview the EEG signal and remove data with obvious drift; S202: Use notch filtering technology to remove 50Hz line noise, and then perform bandpass filtering between 0.1Hz and 45Hz to remove noise that is too high or too low in frequency; S203: changing the original reference electrode FCz to an average reference to optimize the signal reference mode; S204: Decompose the EEG signal into several independent components through independent component analysis (ICA), and combine it with visual inspection to accurately identify and remove artifacts, thereby effectively eliminating noise components.

4. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S2, the specific steps of calculating the inter-subject correlation (ISC) value of each segment include: First, for each pair of subjects k and l, the cross-electrode covariance matrix C is calculated kl , which is: , Among them, X k (t) and X l (t) represent the EEG signals of individuals k and l at time point t, μ k and μ l are the average values of individuals k and l over the entire time series, respectively, and T represents the total number of time points of the signal; Then, the following formula was applied to calculate the inter-subject correlation (ISC) value between each pair of subjects: ， Among them, V i represents the i-th eigenvector extracted by PCA or other methods, which is used to capture the key features of the signal; T is the total number of time points of the signal.

5. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S3, the specific steps of constructing the intra-brain network and the inter-brain network by constructing the functional connectivity matrix are as follows: First, the circular correlation coefficient CCorr is selected as the synchronization indicator: ， in, and Represents the phase values of the two channels being compared and the average phase of the two signals, k represents the kth data point, N represents the total number of data points, and Represents the phase value of the kth data point of the two channels; Then, the two subjects used a cap equipped with 32 electrodes, and the EEG signals of the two subjects were merged into a two-dimensional array of 64 electrodes. The functional connectivity matrix was constructed using correlation. The nodes in the matrix represent network nodes, and the values represent the weights of the connections between nodes. The first 32 electrodes were the electrodes in the brain of the first subject, and the back 32 electrodes were the electrodes in the brain of the second subject. CCorr was calculated between the electrodes in the brain of the first subject to obtain the brain network of the first subject, CCorr was calculated between the electrodes in the brain of the second subject to obtain the brain network of the second subject, and CCorr was calculated between the electrodes in the brain of the first subject and the electrodes in the brain of the second subject to obtain the inter-brain network.

6. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S3, the similarity of the brain network is calculated using the following formula: ， Among them, C 1,k,t represents the edge weight of the first node pair (k, t) in the functional connectivity matrix, C 2,k,t represents the edge weight of the second node pair (k, t) in the functional connection matrix, Ne is the number of nodes; SimiNet∈[0,1], 0 and 1 indicate that the two connection matrices are dissimilar and exactly the same, respectively.

7. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S3, the global efficiency of the inter-brain network is calculated using the following formula: ， Where n is the total number of nodes in the network, N is the set of nodes, and d ij is the shortest path length between node i and node j.

8. The method for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 1, characterized in that: In step S3, under the same stimulus, the higher the similarity of the network structure between the two subjects’ brains, the stronger the synchronization between the multiple brains; The global efficiency of cross-brain networks is used to measure the efficiency and stability of information transmission between subjects' brain networks. The higher the global efficiency, the stronger the integration ability of the inter-brain network and the higher the coordination ability of different electrodes or brain regions, which in turn indicates high synchronization.

9. A system for detecting interpersonal neural synchronization under audiovisual stimulation, characterized in that: include: The audiovisual stimulation database construction module is used to design a two-person hyperscanning experimental paradigm, collect two-person EEG signals, and build a database of visual and auditory stimulation; An EEG data preprocessing module is used to preprocess the data in the database to obtain EEG signals in four different frequency bands: δ, θ, α, and β. The EEG signal in the alpha frequency band is extracted and segmented to extract EEG signals under visual and auditory tasks. An inter-subject correlation calculation module is used to divide the EEG signals preprocessed by the EEG data preprocessing module into several segments according to the time of the stimulation material, calculate the inter-subject correlation (ISC) value of each segment, and analyze the trend and stability of the subject correlation over time; by calculating the ISC value of the entire stimulation process for the EEG signals of two subjects during each task, the similarity of brain activity between subjects under the same task or stimulation is evaluated; The brain network construction and evaluation module is used to construct intra-brain networks and inter-brain networks by constructing functional connectivity matrices, and calculate the similarity of intra-brain networks and the global efficiency of inter-brain networks, so as to effectively evaluate the degree of collaboration and synchronization between neural activities between subjects under visual and auditory stimulation.

10. The system for detecting interpersonal neural synchronization under audiovisual stimulation according to claim 9, characterized in that: The step of preprocessing the data in the database by the EEG data preprocessing module includes: First, the EEG signals were previewed and data with obvious drift were removed; Secondly, notch filtering technology is used to remove 50Hz line noise, and then band-pass filtering is performed between 0.1Hz and 45Hz to remove noise that is too high or too low in frequency; Next, the original reference electrode FCz was changed to an average reference to optimize the signal reference method; Finally, the EEG signal is decomposed into several independent components through independent component analysis (ICA), and combined with visual inspection, the artifacts are accurately identified and removed, thereby effectively eliminating the noise component.

Images