Data processing method, display method and device for obtaining cognitive consciousness state intermediate parameters based on heart-brain coupling phase difference mode

Abstract

The patent relates to a data processing method, a display method and a device for acquiring intermediate parameters of a cognitive awareness state based on a heart-brain coupling phase difference mode, which are used for assisting in judging the cognitive awareness state. According to the method, ECG (electrocardiosignal) and EEG (electroencephalogram) are collected and preprocessed, a heart rate variability (HRV) signal and a power spectrum signal of the EEG are extracted, a symbolization mode of instantaneous phase difference of the two signals is analyzed, and a heart-brain coupling index is calculated. The method is suitable for subjects of different ages, and can be applied to the fields of driver cognitive state monitoring, anesthesia depth evaluation, film and television culture work user cognitive consciousness state evaluation and the like. The system is provided with a real-time and non-real-time offline monitoring interface, a quick response interface and a visual output interface, and support is provided for decision making of operators.

Description

Technical Field

[0001] The present invention relates to the field of multimodal physiological signal analysis, and particularly to a data acquisition, processing and display method and device for obtaining intermediate parameters in different cognitive awareness states based on the coupling relationship between cardiac and cerebral activities. Background Art

[0002] As an emerging evaluation method, cardio-cerebral coupling has the advantage of being able to quantify the interaction between the brain and the heart and provide more comprehensive physiological information. Research shows that after the physical and mental states change, the low-frequency coupling between the heart and the brain will increase, and the cardio-cerebral coupling index has a significant improvement in prognostic performance. However, in the field of cognitive awareness state evaluation research, the application of cardio-cerebral coupling is not sufficient, which makes it difficult to explain and understand the role of cardio-cerebral coupling. Especially when assisting in making decisions about the cognitive awareness state of subjects, a more transparent and interpretable cardio-cerebral coupling quantification evaluation method is needed. In addition, there is currently a lack of a full-process dynamic fusion monitoring process for cardio-cerebral coupling monitoring, and it is difficult to be generally applicable to accurately predicting different cognitive awareness stages of subjects in various situations. Therefore, developing a cognitive awareness state evaluation method that integrates the dynamic changes of cardio-cerebral coupling can not only improve the accuracy of monitoring different cognitive awareness states, but also provide personalized cognitive awareness state monitoring for subjects of different ages.

[0003] The acquisition of relevant parameters based on cardio-cerebral coupling can effectively assist in the evaluation of cognitive awareness states and can be applied to the auxiliary judgment of intermediate states in multiple fields. For example, in the field of traffic safety, the method of the present invention can assist in determining whether a driver is in a state of intoxication or drug use, so as to provide a highly real-time non-invasive inspection for the daily inspection of airplane and high-speed rail drivers and reduce the occurrence of traffic safety accidents; in addition, it can also be used for traffic accident identification to determine whether a traffic safety accident is caused by a driver's drug use or intoxication after the accident. For example, in the field of anti-doping detection in sports events, the intake of doping may cause the athlete's body to be in an abnormally excited state, resulting in abnormal cardio-cerebral coupling patterns. Therefore, the method of the present invention can be used to assist in prompting different states of athletes from the onset to the disappearance of the effect of doping intake to assist in anti-doping detection. Another example is in the fields of game, movie and advertisement recommendation. Obtaining relevant cardio-cerebral coupling index parameters can effectively assist in judging the psychological cognitive states of relevant audience groups to improve the user experience of relevant film and television entertainment works.

[0004] For another example, general anesthesia plays a crucial role in modern medicine. Its core purpose is to ensure that the anesthetized individual is pain-free and unconscious during the operation, thus avoiding the physical and psychological traumas that may be caused by the surgical operation. However, there are significant differences in the responses of individuals of different ages to anesthetic drugs, which makes it particularly urgent to develop an anesthesia depth assessment index applicable to people of all ages. In addition, if only a single electroencephalogram (EEG) signal is used as the main monitoring means for the anesthesia state, it is easy to misjudge the anesthesia stage of the individual because of the different sensitivities of individuals to various anesthetic drugs, resulting in the same EEG signal value but actually being in different anesthesia stages. In the field of anesthesia monitoring, EEG and electrocardiogram (ECG) signals are key physiological parameters for evaluating anesthesia depth. The EEG signal can directly reflect the neural activities and consciousness state of the brain. Under anesthesia, the changes in EEG can intuitively show the inhibitory degree of anesthetic drugs on the brain and the changes in functional connectivity when the brain consciousness state changes. And the heart rate variability (HRV) derived from the ECG signal can reflect the activities of the autonomic nervous system, especially the balance state of the sympathetic and parasympathetic nerves. Under anesthesia, the changes in HRV can reveal the balance changes between nociception and analgesia, as well as the effects of anesthetic drugs on the autonomic nervous system. Therefore, the combination of EEG and ECG signals can comprehensively monitor the intraoperative anesthesia state at two levels: the neural activities of the brain and the activities of the autonomic nervous system. Summary of the Invention

[0005] The present application provides a data processing method, a display method and a device for obtaining intermediate parameters of cognitive awareness state based on the cardio-cerebral coupling phase difference mode, so as to assist in realizing a monitoring and evaluation scheme for the cognitive awareness stage of subjects of different ages, which is intuitive, highly interpretable, accurate and capable of rapid calculation.

[0006] In the first aspect of the technical solution of the present application, the present application provides a data processing method for obtaining intermediate parameters of cognitive awareness state based on the cardio-cerebral coupling phase difference mode, and the method includes the following steps:

[0007] Extracting the electrocardiogram (ECG) signal and the electroencephalogram (EEG) signal of the subject during different cognitive awareness states based on a multimodal physiological signal acquisition system, and marking and storing different cognitive awareness states during the acquisition process for subsequent analysis;

[0008] Preprocessing the electrocardiogram signal and the electroencephalogram signal extracted by the multimodal physiological signal acquisition system, and extracting the corresponding characteristic segments of the electrocardiogram signal and the electroencephalogram signal; Optionally, the electroencephalogram signal and the electrocardiogram signal in the range of 0.1 - 45 Hz can be obtained through preprocessing, and the multimodal physiological signal can be intercepted into two-minute segments in different states;

[0009] Perform R peak detection on the preprocessed electrocardiogram (ECG) signal, and obtain heart rate variability (HRV) data through the variability of the continuously measured RR interval sequence;

[0010] Adopt fast Fourier transform to convert the time-domain electroencephalogram (EEG) signal to the frequency domain and calculate the power spectrum of the EEG signal under various cognitive awareness states;

[0011] Based on the above-mentioned ECG-based HRV signal and EEG power spectrum signal, analyze the symbolic pattern changes of the instantaneous phase difference time series of the two signals to obtain the heart-brain coupling characteristic index;

[0012] Optionally, the power spectrum signal of the EEG is the power spectrum signal based on the alpha (8 - 13 Hz) frequency band;

[0013] Based on the heart-brain coupling characteristic index, determine different cognitive awareness states;

[0014] In the above technical solution, further, the various cognitive awareness states include various states such as sober, drunk, and recovery in the drunk stage;

[0015] In the above technical solution, further, the various cognitive awareness states include various states such as sober, excited, and recovery in the stage of taking stimulants;

[0016] In the above technical solution, further, the various cognitive awareness states include various states such as peaceful, sad, angry, fearful, focused, bored, fatigued, and excited under external stimuli;

[0017] In the above technical solution, further, the various cognitive awareness states include various states such as sober, anesthetized, and recovery states in the anesthesia stage;

[0018] In the above technical solution, further, the preprocessing operations of the ECG signal and EEG signal include filtering, artifact removal, and baseline drift data;

[0019] In the above technical solution, further, the R peak detection is performed by using the parabola fitting method. Fit the parabola equation to each data point of the ECG signal, calculate the polynomial coefficient k by minimizing the quadratic error criterion, screen the R peak direction according to the positive and negative of the k value, eliminate the peaks with k > 0, calculate the parabola height L, and set the threshold through statistical indicators to screen the R peak points, and generate the R - R interval sequence based on the screened R peak points.

[0020] Optionally, the best parabola for fitting any point of the ECG signal is:

[0021] y(n) = k(a - n) 2 + y1(a)

[0022] Here, \(n = a - \tau, L, a - 1, 1, a + 1, L, a + \tau\). The analysis window \(W\) for each parabola fitting is \(2\tau + 1\).

[0023] Optionally, the polynomial coefficient solving method calculates the polynomial coefficient \(k\) by minimizing the quadratic error criterion:

[0024]

[0025] where the error \(e(n)=y(n)-y_1(n)\) such that the derivative of \(V(k)\) That is:

[0026]

[0027] The polynomial coefficient \(k\) is obtained by solving the above formula, where the constant is obtained:

[0028]

[0029] where \(k \lt 0\) indicates that the peak direction is upward, and \(k \gt 0\) indicates that the peak direction is downward;

[0030] Optionally, the height \(L\) of the parabola is used to detect the R peak value, where \(L = |y(a)-y(a - \tau)|\), and we get: \(L = |k|\tau\) 2 .

[0031] Optionally, the specific process of monitoring the R peak by the best parabola fitting algorithm is as follows:

[0032] Calculate all \(k\) values of each segment of the ECG signal, eliminate all cases where \(k \gt 0\), and calculate all corresponding values according to the obtained \(k\) values;

[0033] Obtain the threshold for determining the best value. As a further preferred embodiment, since the ECG signal contains wave peaks such as P wave, R wave, T wave, and U wave, so let it be equal to the upper quartile of all found values, and all points are used as alternative R peak points;

[0034] According to the heart rate value, set the corresponding window and step parameters. Search for the best R peak point, and the maximum value within the window is the R peak point. As a further preferred embodiment, set the window to 0.5 seconds and the step to 0.1 seconds.

[0035] In the above technical solution, further, an R-R interval sequence is extracted from the R peak points, and finally the HRV signal is calculated.

[0036] In the above technical solution, further, the specific calculation process of the heart-brain coupling characteristic index is as follows:

[0037] Resample the HRV signal calculated from the electrocardiogram signal ECG and the power spectrum signal of the alpha (8 - 13 Hz) band of EEG to time series of the same length with a sampling rate of 2 Hz; obtain the phase difference between the two signals through Hilbert transform; when the phase of the HRV signal leads the EEG power spectrum, mark it with the symbol "+1"; when the phase of the HRV signal is synchronized with the EEG power spectrum, mark it with the symbol "0"; when the phase of the HRV signal lags behind the EEG power spectrum, mark it with the symbol "-1"; through the above three cases, obtain the symbolized time series of the phase difference as follows:

[0038] X n ={x n ,x n+t ,L,x n+(m-1)t}

[0039] Here, m is the symbol pattern size, and t represents the time delay length. In the present invention, m = 3 and t = 6, then the number of symbolized patterns is species;

[0040] The probability of any symbolized pattern l appearing in all patterns is calculated as follows:

[0041]

[0042] Here, N l represents the number of times pattern l appears, then the heart - brain coupling (HBC) is defined as follows:

[0043]

[0044] Normalize HBC to nHBC, and assist in judging the cognitive awareness state through nHBC.

[0045] Optionally, set different nHBC thresholds of the heart - brain coupling index based on the heart - brain coupling patterns of different age groups of users to assist in judging and prompting different cognitive awareness states, and switch according to multiple working modes to adapt to the corresponding age groups.

[0046] In the second aspect of the technical solution of the present application, the present application provides a device for obtaining intermediate parameters for cognitive awareness state based on the heart - brain coupling phase difference pattern, including an electrocardiogram signal acquisition module, an electroencephalogram signal acquisition module, a signal pre - processing module, an HRV calculation module, a power spectrum extraction module, a resampling module, a phase difference calculation module, a heart - brain coupling index calculation module, a heart - brain coupling index calculation module;

[0047] The electrocardiogram signal acquisition module is used to acquire the electrocardiogram signal (ECG) of the subject;

[0048] The electroencephalogram signal acquisition module is used to acquire the electroencephalogram signal (EEG) of the subject;

[0049] The signal preprocessing module is used to perform filtering, artifact removal, and baseline drift data preprocessing on the ECG signal and EEG signal to remove noise and interference;

[0050] The HRV calculation module is used to calculate the heart rate variability (HRV) signal based on the identified R peaks;

[0051] The power spectrum extraction module is used to extract the power spectrum signal of the EEG signal;

[0052] The resampling module is used to resample the HRV signal and the power spectrum signal of the EEG so that the resampled signals have the same sampling rate and time length;

[0053] Optionally, the power spectrum signal of the EEG signal is the alpha band power spectrum signal;

[0054] The phase difference calculation module is used to calculate the phase difference data between the HRV signal and the EEG signal through Hilbert transform and perform symbolic processing on the phase difference data to obtain a symbolic time series of the phase difference;

[0055] The heart-brain coupling index calculation module is used to count the occurrence probabilities of different symbolic patterns in the symbolic time series of the phase difference and calculate the heart-brain coupling index (nHBC value);

[0056] The heart-brain signal coupling matching module is used to determine the heart-brain signal coupling matching mode according to the heart-brain coupling index.

[0057] In the above technical solution, further, the system further includes a parabola fitting module for performing parabola fitting on the preprocessed ECG signal to calculate the peak position of the R wave; the parabola fitting module includes:

[0058] A polynomial coefficient calculation unit for performing parabola fitting on each segment of the ECG signal and calculating the polynomial coefficient k of the ECG fitting signal;

[0059] A peak direction judgment unit for judging the direction of the peak according to the positive or negative value of the k value.

[0060] In the above technical solution, further, the heart-brain signal coupling matching module includes:

[0061] A threshold comparison unit for assisting in judging the cognitive awareness state of the subject according to the comparison between the heart-brain coupling index value and a preset threshold;

[0062] A cognitive awareness state parameter output unit for outputting the calculation result of the cognitive awareness state parameter.

[0063] In the above technical solution, further, the system further includes an information display module for displaying relevant cardio-cerebral coupling information, and the information display module includes:

[0064] A user interface module for interacting with the operating user, receiving subject information and test item information input by the user, and displaying intermediate parameters for assisting in cognitive state determination and recommended results;

[0065] An information input unit for inputting basic information of the subject, including name, age, gender, etc.;

[0066] A test item information input unit for inputting relevant information of the test item, including the name of the test item, the setting mode of the test item, etc.;

[0067] A real-time data display unit for real-time displaying the EEG power spectrum, ECG, and HRV waveforms, where the R peak points are marked on the HRV waveform;

[0068] A cognitive state assistance prompt display unit for displaying the cardio-cerebral coupling index and its trend graph, using different colors to represent different cognitive states;

[0069] An alarm prompt unit: for issuing an alarm prompt when the cognitive state is abnormal, reminding the operator to take measures in time.

[0070] In the third aspect of the technical solution of the present application, the present application provides an information display method applicable to a data processing method for obtaining intermediate parameters of cognitive awareness state based on the cardio-cerebral coupling phase difference mode. After obtaining user information data such as age, the user information is displayed on the current display interface. The ECG and EEG signals are obtained in real time and displayed on the display terminal. The EEG power spectrum signal, HRV waveform signal, cardio-cerebral coupling index signal, and their trend indicators are calculated in real time and respectively displayed on the screen. In response to the selected working mode, the cognitive awareness state of the subject is determined, and different colors are used to represent different cognitive awareness states.

[0071] The beneficial effects of this invention application are as follows: By using the R-peak detection method based on parabola fitting and symbolic parameters, the relevant operation speed is accelerated, and the real-time monitoring performance of the system is improved; By adapting the heart-brain coupling mechanism for patients of different ages, a precise quantitative basis is provided for formulating a personalized cognitive awareness state monitoring plan; Thus, the monitoring effect is optimized, and the problems of poor subject compliance caused by invasive monitoring in the past or limited usage scenarios due to the large size and high cost of large MRI or X-ray equipment are reduced; At the same time, an instant feedback is provided to the operator through the visualization interface to assist in making precise decisions during the change of the subject's cognitive awareness state; Through the analysis of the symbolic phase difference pattern, the instantaneous phase synchronization relationship between the heart and brain signals is quantified, and at the same time, an interpretable calculation method for the heart-brain coupling index is provided, which can intuitively judge the central nervous consciousness state and autonomic nerve balance of the subject in different cognitive awareness state stages, and assist in formulating more precise cognitive awareness state monitoring decisions.

[0072] The implementation of the device of this invention can quickly and accurately process the personalized cognitive awareness state monitoring data of subjects of different ages, and can reduce the decision-making time cost through the relevant visualization interface. In addition, the measurement method provided by this invention can reduce the system operation time and hardware resource consumption, and improve the real-time performance of the system operation. Brief Description of the Drawings

[0073] Figure 1 It is a method flow chart of a cognitive awareness state assessment method based on heart-brain coupling provided by an embodiment of this invention;

[0074] Figure 2 It is an effect diagram of extracting R peaks in an embodiment of this invention;

[0075] Figure 3 It is a statistical chart of heart-brain coupling indexes calculated for three patients in a waking state provided by an embodiment of this invention;

[0076] Figure 4 It is a statistical chart of heart-brain coupling indexes calculated for three patients in an anesthetic state provided by an embodiment of this invention;

[0077] Figure 5 It is a statistical chart of heart-brain coupling indexes calculated for three patients in a recovery state provided by an embodiment of this invention;

[0078] Figure 6 It is a statistical comparison chart of heart-brain coupling indexes of the whole process for three patients in a waking, anesthetic, and recovery state respectively provided by an embodiment of this invention;

[0079] Figure 7 It is a visualization output interface provided by an embodiment of this invention;

[0080] Figure 8The flowchart of a data processing method for obtaining intermediate parameters of cognitive awareness state based on the heart-brain coupling phase difference mode that can utilize different data acquisition methods provided by an embodiment of the present invention. Detailed implementation manners

[0081] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0082] It should be noted that although the logical order is shown in the embodiments of the present application, in some cases, the steps shown or described may be executed in a different order from that in the present application. Terms such as "first", "second" and the corresponding numbers in the description, claims and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0083] Figure 1 A data processing method for obtaining intermediate parameters of cognitive awareness state based on the heart-brain coupling phase difference mode provided by an embodiment of the present invention is shown. The method can be applied to the field of anesthesia state monitoring. The method applied to the field of anesthesia state monitoring includes the following steps:

[0084] S101: Synchronously collect the ECG and EEG signal data of the subject in the general anesthesia state, and the process includes;

[0085] Specifically, during the ECG signal collection process, the operator needs to ensure that the subject to be measured is in a relaxed sitting or lying position and maintains a relaxed posture. The operator needs to clean the skin of the measured part of the chest and / or back of the subject to be measured and place multiple electrodes on its surface for capturing the electrical activity of the heart. After the operator places the electrodes, connect the electrodes to the electrocardiograph through wires and set the corresponding parameters such as sensitivity, noise, input impedance and frequency response; the system can also be selected according to the operator;

[0086] Optionally, the different cognitive awareness states of the subject are different processes of waking, anesthesia, and recovery in the anesthesia state. Optionally, this process is induced by propofol and maintained by sevoflurane;

[0087] Optionally, the electrode material is a silver / silver chloride electrode or a platinum electrode or a disposable electrocardiogram electrode;

[0088] Optionally, the number of electrodes during the ECG signal collection process can be selected according to clinical needs, including but not limited to 3-lead, 5-lead, 12-lead, 18-lead and 36-lead;

[0089] Optionally, the operator can set corresponding parameters according to the corresponding situation of the object to be measured. For example, the frequency response threshold of young children (children less than or equal to 6 years old) can be set to 250 HZ to better capture the electrocardiogram characteristics of young children. For school-age children (6 to 12-year-old children), teenagers, and adults, the operator can set the frequency response to 0.05 HZ to 150 HZ to effectively obtain the electrocardiogram signal characteristics of the measured objects of different ages;

[0090] Optionally, the ECG signal acquisition device can select a patient monitor CMS8000 (Contec, Qinhuangdao) with a corresponding liquid crystal display and visualization interface for data acquisition;

[0091] Specifically, during the EEG signal acquisition process, the operator needs to ensure that the subject is in a relaxed sitting or lying position and maintains a relaxed posture. The operator needs to clean the skin of the corresponding measured part of the subject's scalp and place multiple electrodes, and the electrodes are arranged according to the standard electrode position system (such as the international 10-20 system). The electrodes are used to capture the electrical activity of the brain. Common types include dry electrodes, wet electrodes, and adhesive electrodes, and the electrodes can be selected as silver / silver chloride electrodes, platinum electrodes, or other electrodes according to needs;

[0092] As an optional solution, the number of electrodes during the EEG signal acquisition process can be selected according to clinical needs, including but not limited to technical solutions such as 19 leads, 21 leads, 32 leads, 64 leads, and 128 leads;

[0093] As an optional solution, the process can be optimized for young children (children less than or equal to 6 years old), school-age children (6 to 12-year-old children), teenagers, and adults during the EEG signal acquisition process. Specifically, for young children or school-age children, disc electrodes are preferred to reduce skin irritation, enhance the comfort and cooperation of child subjects, and achieve the purpose of stable signal acquisition and error reduction; for young children and school-age children with low cooperation or older subjects with mental retardation and epilepsy, relevant materials such as medical tape can be used for proper fixation; particularly, for young children, the electrode impedance can be appropriately adjusted to be within 10 kΩ to prevent skin irritation; and the corresponding sensitivity can be adjusted according to the characteristics of the higher brain voltage of young children (such as set at 10 μV / mm) to make the information capture more adaptable to the child's situation as needed.

[0094] Optionally, the EEG signal acquisition device can select a patient monitor BIS vista monitor (Aspect, USA) with a corresponding display for data acquisition;

[0095] Optionally, the ECG and EEG signals of the subject to be measured can be synchronously collected at different stages such as waking, anesthesia, and recovery from general anesthesia, and the duration of the collected data can be ensured according to data processing or actual needs, and the corresponding signal data can be stored;

[0096] S102: Preprocess the collected signals and intercept the signal segments, and the process includes;

[0097] Specifically, corresponding filters can be used to eliminate the noise and interference in the ECG and / or EEG signals respectively, and intercept the synchronized ECG and / or EEG signals;

[0098] As an optional technical solution, a Butterworth band-pass filter can be selected to filter the ECG signal. The upper and lower threshold values of the Butterworth band-pass filter bandwidth can be set to 45 Hz and 0.1 Hz respectively to filter the ECG signal and store the processed data;

[0099] As an optional technical solution, an adaptive filter can be selected to remove the power frequency interference from the collected EEG signals. Optionally, a band-pass finite impulse response (FIR) filter is used to eliminate high-frequency noise (>45 Hz) and low-frequency baseline drift (<0.1 Hz), and retain the effective information required for anesthesia depth assessment; The influence of electromyographic signals can be detected and eliminated by an inverse filter, and the EEG intermediate signal can be obtained after removing all electrooculogram signals by wavelet transform; The EEG intermediate signals collected within a certain period of time and processed by removing power frequency interference and / or electrooculogram signals and electromyographic signals can be segmented into multiple data segments at 10 s as the segmentation unit, and the data with an electroencephalogram amplitude exceeding 2002 V or more than 3 standard deviations in each data segment is removed. After all data segments are processed, all processed data segments are merged in the order of acquisition time to obtain the final EEG data after filtering;

[0100] Optionally, the process of segmenting the EEG intermediate signal and removing the data with large amplitudes from multiple data segments can be performed sequentially or in parallel;

[0101] Align the filtered ECG signal and the final filtered EEG signal according to the acquisition data time, and intercept the EEG and ECG data collected and filtered within the synchronization time according to different anesthesia stages such as waking, anesthesia, and recovery from general anesthesia of the measured patient according to a preset time period. The ECG data and EEG data should ensure time synchronization for data acquisition and be in a normal and stable working state;

[0102] Optionally, the preset time period for intercepting the EEG and ECG data collected and filtered within the synchronization time can be 2 minutes;

[0103] As an alternative implementation, the time synchronization process can be used to ensure the time alignment of EEG signal acquisition and ECG signal acquisition, which includes at least one of the following methods:

[0104] Realize the time alignment of EEG signal acquisition and ECG signal acquisition through hardware-level timestamp synchronization (such as the LabStreamingLayer protocol);

[0105] Realize the time alignment of EEG signal acquisition and ECG signal acquisition through the vertical synchronization signal (VSS).

[0106] As a preferred embodiment, the hardware-level timestamp synchronization technology can adopt the LabStreamingLayer protocol. The hardware-level timestamp synchronization technology creates a LabStreamingLayer data stream and generates timestamps through software and hardware configurations that adapt to the LabStreamingLayer protocol. After obtaining the corresponding data stream, data integration is performed through relevant tools to obtain timestamp alignment data, and data fusion processing is performed on ECG and EEG data with different sampling rates as needed.

[0107] The process of software and hardware configuration adapting to the LabStreamingLayer protocol includes at least one of the following steps:

[0108] Select devices with LabStreamingLayer compatibility to collect ECG and EEG data and directly generate LabStreamingLayer data streams;

[0109] Select devices to collect ECG and EEG data and generate LabStreamingLayer data streams through plug-ins / driver programs;

[0110] Select devices to collect ECG and EEG data and generate LabStreamingLayer data streams through third-party integration tools.

[0111] Optionally, all devices configured with software and hardware adapting to the LabStreamingLayer protocol are connected to the same local area network (LAN or WAN), and NTP-like time synchronization algorithms are used to eliminate network latency differences. The device access method can be selected as wired access to reduce the packet loss risk in a wireless environment.

[0112] The data stream creation process generates independent LabStreamingLayer data streams by the device. The timestamp generation process sends a stimulus trigger signal as a time alignment reference by software, and attaches local clock timestamps to each data point by the LabStreamingLayer data stream, and realizes multi-device time synchronization through dynamic clock offset calibration. The software for sending the stimulus trigger signal can be selected from PsychoPy, Unity, etc. The stimulus trigger signal can select the moment when the target image is displayed as the trigger signal.

[0113] The data integration process saves them as specific files by the corresponding integration tool. Optionally, the integration tool is LabRecorder. Optionally, the specific file is an XDF format file.

[0114] The ECG and EEG data fusion process is that a specific tool or plugin parses the specific file, extracts the corresponding ECG and EEG data streams and timestamps, and performs downsampling or interpolation processing based on the different sampling rates of ECG and EEG to generate a multi-modal data set with a unified time baseline. Optionally, the specific tool is the XDF Tools or the Mobilab plugin of EEGLAB. Optionally, the specific file is an XDF format file.

[0115] As a preferred embodiment, the vertical synchronization signal (VSS) realizes the time alignment of EEG signal and ECG signal acquisition by using the vertical synchronization signal as a global time reference, and realizes the time alignment of EEG and ECG signals through hardware trigger and software timestamp marking. The process of the vertical synchronization signal (VSS) realizing the time alignment of EEG signal and ECG signal acquisition ensures the synchronization of the two signals within millisecond-level accuracy through a synchronization trigger mechanism, a delay compensation algorithm and multi-modal data fusion.

[0116] The hardware trigger process uses dedicated hardware to generate a stable vertical synchronization signal VSS pulse signal and sends the VSS signal to the trigger input ports of EEG and ECG acquisition devices simultaneously through a signal distribution circuit pulse to eliminate the signal transmission path difference. Optionally, the dedicated hardware is an FPGA or a dedicated clock chip. Optionally, the signal distribution circuit is an optocoupler isolator.

[0117] The software timestamp marking process records the current sampling point numbers and absolute times of EEG and ECG devices when each VSS pulse triggers.

[0118] The synchronization trigger mechanism includes an initialization stage and a periodic calibration stage. In the initialization stage, all EEG and ECG sampling units are reset by sending VSS pulses to ensure that each channel starts sampling from the same time origin. In the periodic calibration stage, the clock cumulative error is compensated by sending a VSS pulse once within a certain calibration period. Optionally, the calibration period can be 10 seconds.

[0119] The delay compensation algorithm corrects the timestamp by comprehensively considering the group delay of the analog filter and the delays in analog-to-digital conversion and data transmission. Optionally, the delay compensation algorithm can be corrected by the following formula:

[0120] t_corrected = t_raw - τ_filter - N_sample / f_s

[0121] Where t_corrected is the true event occurrence time after delay compensation; t_raw is the original uncompensated timestamp; τ_filter is the phase delay generated by the corresponding filter such as a band-pass filter during the acquisition of ECG or EEG signals. Optionally, the typical phase delay of the EEG is 5 - 20 ms; the typical phase delay of the ECG signal can be set to 3 - 15 ms; N_sample / f_s is the delay in analog-to-digital conversion and data transmission during signal acquisition, 1 / f_s is the time of each sampling period, and N_sample is the number of sampling period delays generated by analog-to-digital conversion and data transmission. Optionally, the number of sampling period delays generated by analog-to-digital conversion and data transmission can be set to 1 to 2 sampling periods.

[0122] The multi-modal data fusion process constructs a global time axis based on the VSS pulse and uses the interpolation method to align signals with different sampling rates.

[0123] S103: Extract the spectral features of the collected EEG signals, which is characterized by the following steps;

[0124] The EEG time-domain signal is converted to a frequency-domain representation through decomposition by the orthogonal basis function method. The orthogonal basis function method projects and transforms the EEG time-domain signal by selecting an orthogonal basis function set, and maps the signal energy distribution to a preset frequency band range;

[0125] Based on the expansion coefficients of the basis functions, calculate the power spectral density, phase distribution, and inter-band coupling parameters of each frequency component, and construct a multi-dimensional frequency-domain feature vector;

[0126] According to the spatio-temporal evolution law of the frequency-domain feature vector, establish a mapping model with the target physiological state.

[0127] Optionally, the orthogonal basis function method is a linear orthogonal transformation, which includes at least one of the following methods: fixed basis functions such as Fourier transform (FFT / DFT), cosine transform (DCT), etc.;

[0128] Optionally, the orthogonal basis function method is a joint time-frequency analysis, which includes at least one of the following methods: time-varying window functions such as short-time Fourier transform (STFT), wavelet transform (Wavelet), etc.;

[0129] Optionally, the orthogonal basis function method is an adaptive decomposition method, which includes at least one of the following methods: data-driven methods such as empirical mode decomposition (EMD), variational mode decomposition (VMD), etc.

[0130] FFT is a fast algorithm for the discrete Fourier transform (DFT). It utilizes the symmetry and periodicity of the complex exponential function to reduce the computational amount. The most commonly used FFT algorithm is the Cooley-Tukey algorithm, which is based on the divide-and-conquer strategy, decomposes the DFT into smaller DFTs, and then recursively calculates these smaller DFTs.

[0131] As a specific embodiment, the following takes the common fast Fourier transform (FFT) of the Cooley-Tukey algorithm as an example to illustrate the specific process of extracting the spectral characteristics of the EEG signals collected by the orthogonal basis function decomposition. The specific process of extracting the spectral characteristics of the collected EEG signals based on the Cooley-Tukey algorithm includes a recursive decomposition and butterfly operation process and a power spectrum calculation and frequency domain feature extraction process.

[0132] Optionally, the recursive decomposition includes a radix-2 decomposition process, and the radix-2 decomposition includes the following steps:

[0133] Divide the input sequence, and divide the input sequence x[n] into two groups by decimation-in-time (DIT):

[0134] Even-order subsequence: x[2k],

[0135] Odd-order subsequence: x[2k + 1];

[0136] Perform recursive FFT calculations. Perform an N / 2-point FFT on the even-order subsequence to obtain X_even[k], and perform an N / 2-point FFT on the odd-order subsequence to obtain X_odd[k];

[0137] Perform result synthesis. Synthesize the final result through butterfly operations. The formula is as follows:

[0138] X[k] = X_even[k] + W_N^k * X_odd[k],

[0139] X[k + N / 2] = X_even[k] - W_N^k * X_odd[k],

[0140] Define the rotation factor as:

[0141] W_N^k = e^(-j2πk / N).

[0142] Optionally, the butterfly operation process can be optimized by using the complex multiplication simplification and SIMD simplification methods;

[0143] The complex multiplication simplification process reduces the number of complex multiplications by 50% by using the symmetry of the rotation factor. The formula is as follows:

[0144] W_N^(k + N / 2) = -W_N^k

[0145] The SIMD simplification method vectorizes the addition and subtraction operations in the butterfly operation in 4 directions.

[0146] The power spectrum calculation and frequency domain feature extraction process includes the amplitude spectrum and power density calculation and the frequency band energy integration process.

[0147] The amplitude spectrum and power density calculation process includes the complex to amplitude calculation process and the power spectral density (PSD) calculation process;

[0148] The complex to amplitude calculates the modulus value of each point of the FFT output result X(k). The formula is as follows:

[0149] |X(k)| = sqrt(Re{X(k)}^2 + Im{X(k)}^2);

[0150] The formula for the power spectral density (PSD) is as follows:

[0151] P(k) = |X(k)|^2 / (N * F_s * S_window),

[0152] where N is the number of FFT points, F_s is the sampling rate (unit: Hz), and S_window is the window function energy compensation factor.

[0153] The frequency band energy integration can be applied to each typical EEG frequency band, and the typical EEG frequency bands include the δ frequency band (1 - 4 Hz)

[0154] , the θ frequency band (4 - 8 Hz), the α frequency band (8 - 12 Hz), the β frequency band (12 - 30 Hz), and the γ frequency band (30 - 100 Hz).

[0155] The formula for the frequency band energy integration is:

[0156] $E_{band}=\sum_{k = f_l}^{f_h}[P(k)*\Delta f]$;

[0157] Where: $\Delta f = F_s / N$ (frequency resolution); $f_l / f_h$: frequency indices corresponding to the lower / upper limits of the frequency band.

[0158] S104: Extract the R peak points of the collected ECG signals, and the process includes:

[0159] S31. Use the parabola fitting method to detect the R peak of the ECG. The best parabola for fitting any point of the ECG signal is

[0160] $y(n)=k(a - n)$ 2 $+y1(a)$

[0161] Here $n = a - \tau, L, a - 1, 1, a + 1, L, a + \tau$. The analysis window $W = 2\tau + 1$ for each parabola fitting.

[0162] S32. Calculate the parabola polynomial coefficient $k$, and the steps are as follows:

[0163] Calculate the polynomial coefficient $k$ by minimizing the quadratic error criterion:

[0164]

[0165] Where the error $e(n)=y(n)-y1(n)$, making the derivative of $V(k)$ That is:

[0166]

[0167] Solve the above equation to obtain the polynomial coefficient $k$, where let the constant Get:

[0168]

[0169] Where $k\lt0$ indicates that the peak direction is upward, and $k\gt0$ indicates that the peak direction is downward;

[0170] S33. Use the height of the parabola to detect the R peak, where $L = |y(a)-y(a - \tau)|$, and get:

[0171] $L = |k|\tau$ 2 .

[0172] S34. Monitor the R peak through the best parabola fitting algorithm, which includes the following steps:

[0173] S341. Calculate all $k$ values of each segment of the ECG signal, eliminate all cases where $k\gt0$, and calculate all corresponding values according to the obtained $k$ values;

[0174] S342. Obtain the threshold L for determining the optimal L value th . As an alternative implementation, since the ECG signal contains wave peaks such as P wave, R wave, T wave, U wave, etc., let L th be equal to the upper quartile of all the found L values, and all points where L > L th are used as candidate R peak points.

[0175] S343. According to the heart rate value, set corresponding values such as window and step size to find the optimal R peak point. The maximum value within the window range is the R peak point. As an alternative implementation, a window of 0.5 seconds and a step size of 0.1 seconds can be set.

[0176] S105: Calculate the HRV time series, and its process includes;

[0177] Extract the R-R interval sequence through the R peak points, and finally calculate the HRV signal.

[0178] S106: Calculate the coupling characteristics of multi-modal cardiac and brain signals, and its process includes;

[0179] Based on the HRV signal of ECG and the power spectrum signal of the alpha (8 - 13 Hz) frequency band of EEG, by analyzing the symbolic pattern changes of the instantaneous phase difference time series of the two signals, obtain the heart-brain coupling characteristics.

[0180] S41 Resample the ECG and EEG signals to time series of the same length with a sampling rate of 2 Hz;

[0181] S42 Obtain the phase difference of the two signals through Hilbert transform There are the following three cases: namely Then the phase difference symbol x of this point n = 1; Then the phase difference symbol x of this point n = 0 if Then the phase difference symbol x of this point n = -1. S43. Through the above three cases, obtain the phase difference symbolic time series as follows:

[0182] X n = {x n , x n+t , L, x n+(m-1)t}

[0183] Here, m is the symbol pattern size, and t represents the time delay length. In the present invention, m = 3 and t = 6, then the number of symbolic patterns is types.

[0184] S44. Since most data will exhibit multiple different symbolization patterns, define any symbolization pattern as l. The probability of l occurring in all patterns is calculated as follows:

[0185]

[0186] Here, N l represents the number of times pattern l appears. Then, the heart-brain coupling (HBC) is defined as follows:

[0187]

[0188] S55. Normalize HBC to nHBC, and its value range is fixed. The closer the nHBC value is to 1, the less regular the phase difference pattern is, and the probabilities of all patterns appearing are nearly the same, indicating that the phase difference pattern of the heart-brain signal is more diverse; the closer the nHBC value is to 0, the more regular the phase difference pattern is, and only a small number of patterns dominate, indicating a single coupling pattern of the heart-brain signal.

[0189] S107: Analyze the changes in the heart-brain coupling of users in different states. The process includes:

[0190] Calculate the electrocardiogram coupling index nHBC of the subjects in different states respectively, comprehensively compare to obtain the corresponding state intermediate parameters and assist in the corresponding decision-making.

[0191] The following further illustrates the above situation with an example of a subject under general anesthesia, used to determine whether the subject is in three states: awake, recovering, and deep anesthesia.

[0192] Using the method of the present invention, three patients who have undergone general anesthesia surgery are selected as relevant examples for analysis, namely: a 4-year-old pediatric patient 1, female, with the operation name of redundant toe resection; a 12-year-old adolescent patient 2, male, with the operation name of open reduction and internal fixation of radius fracture; a 27-year-old adult patient 3, male, with the operation name of arthroscopic anterior cruciate ligament reconstruction of the knee joint. The results are as Figure 3 、 Figure 4 、 Figure 5 、 Figure 6 shown. Among them Figure 3 shown is the statistical chart of the heart-brain coupling index calculated based on the method of the present invention for the three patients when they are awake. Figure 4 is the statistical chart of the heart-brain coupling index calculated based on the present invention for the three patients when they are under anesthesia. Figure 5 is the statistical chart of the heart-brain coupling index calculated based on the present invention for the three patients when they are in the recovery state. Figure 6It is a statistical comparison chart of the whole-process cardio-cerebral coupling indexes of three patients in the awake, anesthetized, and recovery states. In the embodiment, the value of τ in the present invention is 25 data points, and the width W of the parabola opening is 51. During anesthesia, the nHBC value is significantly lower than that in the awake and recovery periods, which indicates that the nHBC value can effectively distinguish the anesthetic state of the patient. In addition, the high nHBC value in the awake state also shows that the phase difference mode of cardio-cerebral coupling is more diverse in the awake state. Due to the action of anesthetic drugs, the patient's consciousness level decreases, the action of inhibitory neurotransmitters (such as gamma-aminobutyric acid, GABA) is reduced or the action of excitatory neurotransmitters (such as glutamate) is affected, and the autonomous control ability of the heart is affected, ultimately weakening the coupling relationship between the heart and the brain. After the patient wakes up from the anesthetic state, the degree of cardio-cerebral coupling does not completely return to the awake state. As can be seen from the figure, the nHBC value in the recovery state is still slightly lower than that in the awake state, which may indicate that the effect of anesthetic drugs on the heart and brain has not completely disappeared. For different age groups, during the awake and anesthetic periods, the cardio-cerebral coupling of adolescent patients is stronger than that of infant and adult patients. This may indicate that with the continuous development of the heart and brain, in adolescence, the cardio-cerebral coupling develops to its peak. And because it begins to degenerate after reaching the peak, the cardio-cerebral coupling of adults shows a decline, but it is still stronger than that in infancy during anesthesia. During the recovery period, the cardio-cerebral coupling of infant patients is stronger than that of adolescent and adult patients, and is closer to the cardio-cerebral coupling intensity in the awake period, further indicating that different age patients have different responses to general anesthetic drugs. According to age, the general range of the cardio-cerebral coupling nHBC in the awake period is above 0.93, in the anesthetic period is between 0.7 and 0.8, and in the recovery period is between 0.85 and 0.96.

[0193] Figure 7 It is the output interface for the assessment of cardio-cerebral coupling anesthesia depth. The content of the output interface includes: basic patient information, including name, age, etc.; surgical information, including surgical name and anesthetic drugs, etc.; real-time EEG, alpha power spectrum, ECG, and HRV waveforms, where the R peak points are marked on the HRV waveform; and the nHBC index and its trend chart, using different colors to represent different anesthetic states, such as red for the awake period, green for the anesthetic period, and blue for the recovery period.

[0194] The heart-brain coupling anesthesia depth assessment method is an innovative multimodal physiological signal analysis technique that evaluates anesthesia depth by synchronously collecting and analyzing electroencephalogram (EEG) signals and electrocardiogram (ECG) signals. This method utilizes the brain nerve activities and changes in consciousness state reflected by EEG, as well as the autonomic nervous system activities indicated by the heart rate variability derived from ECG, especially the balance state of the sympathetic and parasympathetic nerves. By analyzing the symbolic pattern changes in the time series of the instantaneous phase difference between these two signals, heart-brain coupling characteristics can be obtained, thereby providing more comprehensive and accurate physiological information for the monitoring of anesthesia depth. This method not only improves the accuracy of anesthesia monitoring but also helps to optimize the anesthesia effect and reduce complications.

[0195] In summary, compared with the prior art, the embodiments of the present invention have the following advantages:

[0196] (1) By synchronously collecting EEG signals and ECG signals and analyzing their interactions, the present invention provides a brand-new multimodal physiological signal analysis method. This method can not only monitor the nerve activities and consciousness state of the brain but also reflect the activities of the autonomic nervous system, thereby providing more comprehensive physiological information for the assessment of anesthesia depth.

[0197] (2) The present invention uses the parabola fitting method to detect the R peaks in the ECG signal. This method can more accurately identify the R waves because it takes into account the local waveform characteristics of the ECG signal, thereby improving the accuracy of R peak detection. Since the accuracy of the R-R interval sequence directly affects the results of HRV analysis, the precise R peak extraction method of the present invention can improve the accuracy of HRV analysis, thereby providing more reliable data for the assessment of anesthesia depth.

[0198] (3) The present invention proposes an anesthesia depth assessment method based on heart-brain coupling. This method can quantify the interaction between the brain and the heart, provide more comprehensive physiological information, and more precisely discover the phase synchronization law on the time scale. Compared with the traditional single physiological parameter assessment method, the present invention can more accurately reflect the impact of anesthetic drugs on the central nervous system and the changes in functional connectivity when the brain consciousness state changes.

[0199] (4) The method of the present invention is suitable for real-time monitoring, can quickly respond to changes in the patient's state, and provide timely feedback to clinicians. At the same time, the method of the present invention has good adaptability and can be adjusted according to the specific conditions of different patients to achieve the best monitoring effect.

[0200] (5) The present invention provides a new clinical decision support tool. Through the quantified heart-brain coupling characteristics, doctors can more intuitively understand the patient's state and thus make more accurate clinical decisions.

[0201] Traditional alcohol / drug detection methods have defects such as strong invasiveness (e.g., blood tests) and response lags (e.g., urine tests). By establishing a heart-brain electrical signal coupling model, the present invention can evaluate the functional state of a driver's neurocognitive awareness in real time, providing a new technical means for traffic safety management. As a method for obtaining an intermediate parameter based on the heart-brain coupling phase difference pattern for assisting in judging the cognitive awareness state provided by an embodiment of the present invention, the method can be applied to the field of pre-employment monitoring of drivers. The method applied to the field of monitoring whether a driver is in a drunken or drug-taking state includes the following steps:

[0202] Set non-invasive detection terminals at transportation hubs such as airports and high-speed railway stations. Before a driver takes up a post, through electrocardiogram (ECG) and electroencephalogram (EEG) signal acquisition devices, a preprocessing module is built into the terminal to perform data processing on the collected data, including filtering, artifact removal, baseline drift data, etc.;

[0203] Calculate the heart-brain coupling index based on the multi-modal signal processing process, and calculate the heart-brain coupling index (nHBC value) of the monitored driver in real time; the real-time calculation process uses a parabolic fitting R peak detection ECG algorithm to screen the R wave direction through the positive and negative of the polynomial coefficient k, and combines statistical thresholds to eliminate interference; the real-time calculation process uses the power spectrum of the extracted alpha frequency band (8-13 Hz) to process the EEG signal, uses FFT to convert the frequency domain signal, and removes electrooculogram artifacts through independent component analysis (ICA); the real-time calculation process resamples the HRV and EEG signals to 2 Hz, calculates the phase difference through Hilbert transform, and generates a symbolic time series;

[0204] The real-time calculation process obtains the heart-brain coupling index nHBC through the number of occurrences and probabilities of the symbolic patterns of the symbolic parameters to judge whether the subject (driver) is in a drunken or drug-taking state.

[0205] According to cardiovascular disease risk research, there are significant differences in HRV and brain function among different age groups; therefore, corresponding nHBC thresholds can be set according to the age of the corresponding driver to assist in judging whether the corresponding driver is in different cognitive awareness states such as drunkenness and poisoning.

[0206] Optionally, the following thresholds can be set according to the driver's age to assist in determining and prompting the corresponding drug-taking or drunken state for drivers of different age groups. Optionally, for young people aged 20-39, referring to the heart-brain synchrony of healthy young people, the baseline nHBC threshold pair can be set to 0.85; optionally, for middle-aged and elderly people aged 40-59, considering the degeneration of autonomic nerve function, the threshold can be reduced to 0.78; for drivers who are older or suffer from other neurodegenerative diseases, the threshold can be further adjusted to 0.70.

[0207] When the system detects that the nHBC value continuously falls below a preset threshold (e.g., nHBC < 0.85 for young people), the terminal screen displays a red warning and broadcasts a voice message. At the same time, the data is uploaded to the dispatching center to prompt relevant staff about abnormal cognitive states such as drunkenness.

[0208] The method can also be designed to avoid false alarms. It can be set to trigger an alarm only when the nHBC value continuously falls below the threshold (e.g., for 10 seconds) to avoid transient noise interference. And after the system issues an alarm, on-site staff need to confirm the status through a simple cognitive test (such as reverse digit recall).

[0209] Existing drug / alcohol detection mainly relies on breath analyzers (detecting exhaled alcohol concentration) or saliva / blood tests (detecting drug metabolites), and there are the following problems: Invasive monitoring requires active cooperation in sampling, which is likely to cause resistance (e.g., bus drivers need to frequently undergo saliva tests before going on duty); The timeliness of traditional monitoring is poor. Drug metabolite detection requires laboratory analysis and cannot provide real-time feedback (e.g., the detection lag of marijuana metabolites reaches 4 - 6 hours); Traditional monitoring has limitations in single indicators. It only detects specific substances and cannot evaluate the comprehensive cognitive state (e.g., some unknown sedatives do not exceed the standard but already affect driving ability).

[0210] The advantages of the present invention are that the monitoring method is non-invasive and the compliance of the subjects is good; The present invention has good real-time performance and can provide real-time feedback on the acquisition of monitoring indicators; The present invention is not restricted by specific drugs or specific indicators and can monitor the corresponding cognitive states more comprehensively and robustly.

[0211] Existing evaluation models for the cognitive awareness states of the audience in movie games, virtual reality applications, etc. are relatively single, mainly evaluating the reactions of the audience users through relevant facial functional organs (such as eyes) and corresponding facial expressions. Existing systems mainly judge the cognitive awareness states of users (such as joy, surprise, disgust) by capturing eye movements (such as pupil dilation, gaze direction) and micro facial expressions of facial muscles (such as smiling, frowning) through cameras. However, such technologies are easily affected by environmental light, occlusion (such as VR headsets blocking the face), individual differences in expression habits (such as "poker face"), and cultural background differences (such as cross-cultural ambiguities of emojis), resulting in misjudgments. Existing technologies do not integrate physiological indicators such as ECG signals and electroencephalogram (EEG) to more objectively reflect the true cognitive awareness states of users (such as tension, concentration).

[0212] As a method for obtaining intermediate parameters based on the heart-brain coupling phase difference pattern for assisting in judging cognitive awareness states provided by an embodiment of the present invention, the method can be applied to the field of optimizing the evaluation of the cognitive awareness states of users of film and cultural works. Taking a movie as a demonstration case, the method applied to the field of the cognitive awareness states of users of film and cultural works includes the following steps:

[0213] In a specific screening hall of a cinema, comfortable electrocardiogram (ECG) and electroencephalogram (EEG) signal acquisition devices are provided for the audience to collect the ECG and EEG signals of the audience during the viewing of different types of movies (such as comedies, tragedies, horror movies, science fiction movies, etc.). The signals are preprocessed to remove noise interference caused by external factors such as movie sound effects and picture changes, and the effective signals in the range of 0.1 - 45 Hz are extracted. The signals during the movie viewing process are intercepted into two-minute segments according to different plots or time periods;

[0214] Perform R-peak detection on the ECG signals and calculate HRV data, and at the same time extract the power spectrum of the alpha band of the EEG signals. By analyzing the phase difference symbolization patterns of the two signals, obtain the heart-brain coupling characteristic index, and then determine the mental state of the audience during the movie viewing process, such as joy when watching a comedy, sadness when watching a tragedy, fear when watching a horror movie, and fatigue that may occur when watching a long movie, etc.;

[0215] The movie production side can optimize and adjust aspects such as the plot, rhythm, picture, and sound effects of the movie according to these judgment results. For example, if it is found that the audience generally shows a fatigue state during a certain plot, the duration of this plot can be considered shortened or some interesting elements can be added to attract the audience's attention. In addition, the cinema can also provide more personalized services according to the feedback of the audience's mental state. For example, when the audience feels fear, the volume can be appropriately adjusted or soothing measures can be provided. When the audience feels joy, corresponding interactive activities or souvenirs can be provided to enhance the audience's movie viewing experience and satisfaction.

[0216] This method has the advantages of being not affected by multiple factors such as environmental light factors, individual expression habit differences, and cultural background differences, having reliability and robustness, and being relatively easy to obtain intermediate parameters for assisting in judging the cognitive awareness state based on the heart-brain coupling phase difference pattern according to objective physiological indicators and using them to assist in judging the cognitive awareness state of the test user, thereby improving film and television game entertainment works.

[0217] Figure 8 There is shown a data processing method for obtaining intermediate parameters of cognitive awareness state based on electrocardiogram coupling phase difference and symbolization pattern under different data acquisition methods provided by an embodiment of the present invention. The data processing method includes the following steps:

[0218] S1001: Obtain the ECG and EEG signal data of the subject in different cognitive awareness states, and its process includes;

[0219] Specifically, non-real-time and real-time methods can be selected to obtain the data;

[0220] Optionally, the non-real-time data acquisition method can obtain pre-stored ECG and EEG signal data from a portable storage medium, and the portable storage medium includes portable storage media such as USB Flash Drives, disks, optical discs, and external hard drives;

[0221] Optionally, the non-real-time data acquisition method can obtain pre-stored ECG and EEG signal data from a large-capacity persistent medium, and the large-capacity persistent medium can be a storage medium such as a hard disk drive (HDD) or a tape library;

[0222] Optionally, the non-real-time data acquisition method can obtain pre-stored ECG and EEG signal data from a cache-like medium, and the cache-like medium can be a storage medium such as a non-volatile dual in-line memory module (NVDIMM) or a high-speed solid-state drive (SSD) to temporarily store raw data collected at high frequencies;

[0223] Optionally, the non-real-time data acquisition method can also obtain pre-stored ECG and EEG signal data through distributed cloud storage;

[0224] Optionally, the non-real-time data acquisition method can also use a volatile memory medium to pre-store ECG and EEG signal data, and the volatile memory medium can be a storage medium such as a dynamic random access memory (DRAM) or a static random access memory (SRAM); as an alternative implementation, the ECG and EEG signal data can be transferred from one storage medium to another data storage medium within a preset time; the pre-storage method can perform data collection through a user terminal and then perform local storage or copy and backup to other data storage media. The user terminal can be a professional monitor with EEG and ECG signal acquisition functions, or a wearable device, mobile phone, tablet computer, etc. with corresponding acquisition functions;

[0225] As a preferred technical solution, a wearable device can be selected as the user terminal to temporarily collect ECG and EEG signal data, and the temporarily collected data can be automatically and batch uploaded to a distributed cloud resource node for storage or to a storage medium with a larger storage capacity such as the persistent memory of a mobile phone or tablet to persistently store a large amount of ECG and EEG signal data, preventing data overflow. At the same time, the batch data transmission can be used to reduce the technical drawbacks of low transmission efficiency, low utilization rate, and high energy consumption caused by sporadic data storage. In addition, the computing functions of the cloud resource node or the mobile phone and tablet can be used for corresponding data calculations in the follow-up to reduce the computing load and energy consumption of the wearable device; the preset time can be selected during the charging period to reduce the data and power usage pressure of the wearable device; the preset time can be selected during non-busy periods such as 0:00 to 6:00 in the morning for data transmission, so that the data can be efficiently and quickly transmitted and stored in other media in a smooth network environment. Thus, this embodiment can only execute the data processing process.

[0226] Optionally, the ECG and EEG signal data can be sampled at different sampling rates, and the ECG and EEG signals obtained through different sampling rates can be realigned through technical means such as downsampling and interpolation.

[0227] S1002: Preprocess the collected signals and intercept the signal segments, and the process includes:

[0228] Use corresponding filters to eliminate the noise and interference in the ECG and / or EEG signals respectively and intercept the real-time or non-real-time obtained ECG and / or EEG signals.

[0229] S1003: Extract the spectral characteristics of the collected EEG signals, which is characterized by including the following steps:

[0230] Convert the EEG time-domain signal to a frequency-domain representation through the method of decomposition by orthogonal basis functions. The orthogonal basis function method projects and transforms the EEG time-domain signal by selecting an orthogonal basis function set, and maps the signal energy distribution to a preset frequency band range.

[0231] Calculate the power spectral density, phase distribution, and inter-band coupling parameters of each frequency component based on the expansion coefficients of the basis functions, and construct a multi-dimensional frequency-domain feature vector.

[0232] Establish a mapping model with the target physiological state according to the spatio-temporal evolution law of the frequency-domain feature vector.

[0233] Finally, obtain the power spectral signal of the EEG signal frequency band.

[0234] S1004: Extract the R peak points of the collected ECG signals, and the process includes:

[0235] The polynomial coefficients k are calculated by minimizing the quadratic error criterion, and the R-peaks are detected using the height of the parabola. This is done by calculating all the polynomial coefficient k values for each segment of the ECG signal, screening the corresponding polynomial coefficient k values with the R-peak direction upwards, calculating the height L of the parabola based on the selected polynomial coefficient k values, and finally using the statistical index of L as a threshold to screen the R-peak points.

[0236] S1005: Calculate the HRV time series, and the process includes:

[0237] Extract the R-R interval sequence through the R-peak points, and finally calculate the HRV signal.

[0238] S1006: Calculate the coupling characteristics of multimodal cardiac and brain signals, and the process includes:

[0239] Resample the ECG and EEG signals to obtain time series data of the same length;

[0240] Calculate the HRV signal based on the ECG signal and the frequency band power spectrum signal of the EEG respectively through the time series data;

[0241] Obtain the symbolic parameter time series data by calculating the phase difference between the transformed HRV signal and the EEG frequency band power spectrum signal;

[0242] Obtain the cardiac-brain coupling index and determine the cardiac-brain signal coupling matching mode through the occurrence times and probabilities of the symbolic patterns of the symbolic parameters.

[0243] S1007: Analyze the cardiac-brain coupling changes of the user in different states, and the process includes:

[0244] Calculate the electrocardiogram coupling index of the subjects in different states respectively, comprehensively compare to obtain the corresponding state intermediate parameters and assist in the corresponding decision-making.

[0245] As an optional implementation, all the above steps of the embodiments of the present invention can be fully implemented by hardware. The hardware can be a computer for daily office work, a mobile phone, a tablet computer, a wearable device, etc., or a dedicated computer hardware integration platform such as an FPGA, DSP, ASIC, RISC-V or ARM architecture embedded platform. The hardware can also be a cloud resource computing center and a GPU computing cluster to perform the corresponding calculation acceleration process. Those skilled in the art should understand that the embodiments of the present invention can be partially or fully implemented by a computer program, and the embodiments of the present invention can only execute the data processing process.

[0246] As an alternative embodiment, the present invention does not specifically limit the data acquisition process. The data acquisition process can be the contact acquisition demonstrated in the previous embodiments, or can be based on non-direct contact methods, etc., to collect relevant signals (such as capturing heart rate changes through Wifi signals or sound signals). The signal data acquisition instrument can be a dedicated monitoring instrument, or a wearable device, or other intelligent terminals such as mobile phones and tablets. It can also be a new type of intelligent device such as an intelligent mattress or pillow integrated with relevant signal acquisition functions.

[0247] As an alternative embodiment, the embodiments of the present invention can be applied to the field of scientific research statistics, applied to data generated by the same object at different time periods or data generated by multiple different objects at the same or different time periods, and combined with cloud computing resources for automated large-scale statistical processing. For example, in the transportation field, ECG and EEG data of multiple drivers at different time periods can be collected, and the changes in their cognitive awareness states can be analyzed through cloud computing resources to provide data support for traffic safety research; in the fields of film and television entertainment and human-computer interaction, data of the same audience or a large number of audiences when watching different audio-visual contents can be statistically analyzed to explore the correlation between heart-brain coupling characteristics and emotional responses for relevant audio-visual interaction feedback experience research. In sports research, by analyzing data collected from multiple athletes, their cognitive and physiological changes can be understood to formulate more scientific training research or conduct corresponding compliance inspection research. In particular, when dealing with large-scale data processing, the above research can fully consider means such as anonymization to balance data privacy, data security, and scientific research efficiency. These applications fully demonstrate the wide applicability and high efficiency of the invention in scientific research statistics in multiple fields, providing new analysis means and data support for research in various fields.

[0248] Embodiments implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be obtained through the cloud, or through other signal forms, or through other storage media, or can be in the form of wired or wireless upload or download from other nodes, or can be any other form.

[0249] Those skilled in the art should understand that certain identical or equivalent features included in this embodiment can be combined with those from other embodiments of the present invention to form new embodiments, and the combinations between these different embodiments are still within the protection scope of this application.

[0250] It should be noted that the relevant descriptions in the specification content and the relevant drawings, as well as words such as "including" and "comprising" in the above embodiments, do not exclude the existence of hardware, module units or steps not listed in the claims; corresponding symbolic markings such as S101, S102, and S1001 / S1002 are only for marking purposes to facilitate reading and distinction, and they do not necessarily constitute a front-back logical sequence relationship and can be interpreted as distinguishing marking symbols.

[0251] Those skilled in the art should know that the system for obtaining intermediate parameters of a cognitive awareness state based on a heart-brain coupling phase difference pattern listed in the claims can be implemented by a single integrated hardware or by a distributed hardware system. Those skilled in the art should know that the data processing method for obtaining intermediate parameters of a cognitive awareness state based on a heart-brain coupling phase difference pattern listed in the claims can be partially or fully implemented by a computer program or a computer device, and the method can only execute the data processing process.

[0252] The above is a specific description of the embodiments of the present invention, but the invention of this application is not limited to the above embodiments. Without any creative labor, those skilled in the art can make corresponding adjustments and transformations to, including but not limited to, the setting of specific index thresholds, the configuration of relevant interface colors, the order of relevant program steps, etc., to adapt to different specific scenarios, and these transformations still fall within the protection scope of the claims of this application.

Claims

1. A data processing method for obtaining intermediate parameters of cognitive awareness state based on the phase difference mode of heart-brain coupling, characterized by the following steps: Obtain ECG signals and EEG band signals and perform preprocessing to remove noise and interference. Resample the ECG and EEG signals to obtain time series data of the same length. Calculate the HRV signal based on the ECG signal and the band power spectrum signal of the EEG respectively through the time series data. Obtain the symbolic parameter time series data by calculating the phase difference between the HRV signal and the EEG band power spectrum signal after transformation processing. Obtain the heart-brain coupling index and determine the heart-brain signal coupling matching mode through the occurrence times and probabilities of the symbolic patterns of the symbolic parameters.

2. The method according to claim 1, characterized in that: The ECG signal and the EEG signal are collected based on time synchronization technology. The instantaneous phase difference between the HRV signal and the EEG band power spectrum signal is obtained through Hilbert transform. The instantaneous phase difference modes of the multiple signals are respectively matched with multiple phase difference symbols.

3. The method according to claim 1 or 2, wherein The calculation of the HRV signal of the ECG signal uses the parabola fitting method to detect the R peak to obtain the R-R interval sequence.

4. The method according to claim 3, wherein Calculate the polynomial coefficient k by minimizing the quadratic error criterion and use the height of the parabola to detect the R peak. Calculate all the polynomial coefficient k values of each segment of the ECG signal and screen the corresponding polynomial coefficient k values with the R peak direction upward. Then calculate the height L of the parabola according to the screened polynomial coefficient k values. Finally, use the statistical index of L as the threshold to screen the R peak points.

5. A method for processing electrocardiogram and electroencephalogram signal data according to claim 1, characterized in that, Set different heart-brain coupling index thresholds based on the heart-brain coupling modes of different age groups of users to determine the stage of cognitive awareness state, and switch according to multiple working modes to adapt to the heart-brain coupling modes of the corresponding age groups.

6. An information display method, which can be applied to any of the data processing methods of claims 1-5, characterized in that, After obtaining user information data such as age, display the user information on the current display interface, obtain the ECG and EEG signals in real time and display them on the display terminal, calculate the EEG power spectrum signal, HRV waveform signal, heart-brain coupling index signal and trend index in real time and display them on the screen respectively, match the cognitive awareness state of the subject in response to the selected working mode, and use different colors to represent different cognitive awareness states.

7. An information processing device for obtaining intermediate parameters of cognitive awareness state based on the phase difference mode of heart-brain coupling, comprising: An electrocardiogram signal acquisition module for acquiring the electrocardiogram signal (ECG) of the subject. An electroencephalogram signal acquisition module for acquiring the electroencephalogram signal (EEG) of the subject. A signal preprocessing module for preprocessing the ECG signal and the EEG signal to remove noise and interference. An HRV calculation module for identifying the R peak and calculating the heart rate variability (HRV) signal. A power spectrum extraction module for extracting the EEG signal power spectrum signal. A resampling module for resampling the HRV signal and the EEG power spectrum signal so that the resampled signals have the same sampling rate and time length. A phase difference calculation module, which is used to calculate the phase difference data of the HRV signal and the EEG signal through Hilbert transform, and perform symbolic processing on the data to obtain a symbolic time series of the phase difference; A heart-brain coupling index calculation module, which is used to count the occurrence probabilities of different symbolic patterns in the symbolic time series of the phase difference and calculate the heart-brain coupling index; A heart-brain signal coupling matching module, which is used to determine the heart-brain signal coupling matching pattern according to the heart-brain coupling index.

8. The information processing apparatus for obtaining an intermediate parameter of a cognitive awareness state according to claim 7, characterized in that The information processing device further includes a parabola fitting module, which is used to perform parabola fitting on the preprocessed ECG signal and calculate the peak position of the R wave. The parabola fitting module includes: A polynomial coefficient calculation unit, which is used to perform parabola fitting on each segment of the ECG signal and calculate the polynomial coefficient k of the ECG fitting signal; A peak direction judgment unit, which is used to judge the direction of the peak according to the positive or negative value of the k value.

9. The information processing apparatus for obtaining intermediate parameters of a cognitive awareness state according to claim 7, wherein The heart-brain signal coupling matching module includes: A threshold comparison unit, which is used to assist in obtaining the result of the cognitive awareness state according to the comparison between the heart-brain coupling index and the selected threshold; A cognitive awareness state parameter output unit, which is used to output the calculation result of the cognitive awareness state parameter.

10. The information processing device for obtaining intermediate parameters of the cognitive awareness state according to any one of claims 7-9, characterized in that, The information processing device further includes an information display device, and the information display device further includes: A user interface module, which is used to interact with the user, receive the subject information and test item information input by the user, and display the result of assisting in obtaining the cognitive awareness state; An information input unit, which is used to input the basic information of the subject, including name, age, gender, etc.; A test item information input unit, which is used to input the relevant information of the subject, including the detection item, the corresponding state, etc.; A real-time data display unit, which is used to display the EEG power spectrum, the ECG and the HRV waveform in real time, and mark the R peak point on the HRV waveform; A cognitive awareness state display unit, which is used to display the heart-brain coupling index and the trend chart, and use different colors to represent different cognitive awareness states; An alarm prompt unit: which is used to send an alarm prompt when the cognitive awareness state is abnormal to remind to take measures in time.

Images