Mental state real-time monitoring system based on intelligent wearable device

Abstract

The invention relates to the technical field of mental state monitoring, in particular to a mental state real-time monitoring system based on intelligent wearable equipment, which comprises a physiological signal analysis module, a voice data analysis module, a mental fluctuation evaluation module, a fatigue boundary judgment module and a mental state analysis module. According to the method, by extracting key physiological and voice signal features, the activity stability of the neural signal and the autonomic nervous system of an individual can be more accurately judged, the stability of the physiological signal is effectively evaluated, and by calculating the fluctuation trend of the mental state and combining physiological and voice data, the accuracy of the physiological signal is improved. Real-time changes of emotion fluctuation and mental load of an individual can be captured more comprehensively, so that mental state monitoring is more comprehensive and accurate, real-time evaluation of mental fatigue is performed according to physiological and voice signal changes, a more objective fatigue judgment basis is provided, the accuracy of mental health state evaluation is enhanced, and the mental health state evaluation efficiency is improved. And more accurate mental health state information is provided for individuals.

Description

Technical Field

[0001] The present invention relates to the technical field of mental state monitoring, and particularly to a real-time mental state monitoring system based on intelligent wearable devices. Background Art

[0002] The technical field of mental state monitoring involves the use of physiological signals, behavioral data, and cognitive pattern analysis to evaluate an individual's psychological, physiological, and neurological states. This field includes various detection methods, such as electroencephalogram, heart rate variability, skin conductance response, eye movement tracking, electromyogram signals, etc. Through intelligent sensing devices, algorithm analysis, and data fusion, the quantitative evaluation of mental states such as stress, anxiety, cognitive load, emotional changes, and sleep quality is achieved. This technology is widely used in fields such as medical health, auxiliary diagnosis of mental diseases, psychological assessment, optimization of human-computer interaction, driver fatigue monitoring, etc., providing objective data support for mental health management and physiological state regulation.

[0003] Among them, the real-time mental state monitoring system is used to realize the real-time evaluation and feedback of an individual's mental state based on the fusion analysis of multi-modal physiological signals and behavioral data. The system can collect users' physiological data through wearable devices, environmental sensors, or other biological detection means, and use technologies such as signal processing, pattern recognition, and artificial intelligence to analyze the data to judge key psychological states such as an individual's mental load, emotional fluctuations, and attention level. The system can be used in scenarios such as work stress management, mental health assessment, sports and rehabilitation guidance, intelligent driving safety, game and entertainment interaction, etc., providing accurate mental state monitoring and intervention for individuals and related industries.

[0004] Traditional monitoring systems rely on single physiological signals or behavioral data, lacking comprehensive analysis of multi-signal source data, which limits the accuracy and practicality of monitoring. For example, in the absence of analysis of voice and physiological signals, a single signal source may lead to misjudgment due to environmental interference or individual differences. Traditional systems fail to dynamically evaluate the fluctuations of mental states, resulting in limited capabilities in real-time feedback and immediate intervention. This limitation is particularly prominent in application scenarios that require precise monitoring and real-time response, such as mental health management or driving fatigue monitoring. Existing methods lack sufficient flexibility and adaptability when dealing with complex emotional and mental state changes, making it difficult to provide continuous and effective support in the ever-changing actual application environment. Summary of the Invention

[0005] The purpose of the present invention is to solve the drawbacks existing in the prior art, and to propose a real-time mental state monitoring system based on intelligent wearable devices.

[0006] To achieve the above purpose, the present invention adopts the following technical solution: A real-time mental state monitoring system based on intelligent wearable devices, the system includes:

[0007] Based on the intelligent wearable hat, the physiological signal analysis module acquires electroencephalogram (EEG) signal and heart rate signal data, calculates the RR interval sequence and the electromyogram activity frequency, judges the stability of nerve signals and the activities of the autonomic nervous system, and obtains physiological signal stability information;

[0008] Based on the intelligent wearable hat, the voice data analysis module collects voice signals, extracts pitch data, calculates the speech rate mutation value, records the speech rate change trend within a unit time, calculates the pitch change rate, and obtains the mean deviation of the pitch curve to get voice fluctuation characteristic data;

[0009] Based on the physiological signal stability information and the voice fluctuation characteristic data, the mental fluctuation evaluation module calculates the amplitude oscillation ratio of low frequency to high frequency, combines the heart rate variability cycle and the voice fluctuation characteristics, calculates the mental state fluctuation degree, and obtains the mental state fluctuation trend;

[0010] Based on the mental state fluctuation trend, the fatigue boundary determination module calculates the fluctuation amplitude offset value within a short-time window, judges the stability of brain wave oscillation, calculates the alternation rate of Alpha wave and Theta wave, identifies whether the amplitude change deviates from the resting state reference value, combines the heart rate variability cycle, calculates the mental fatigue index, and obtains the mental fatigue analysis information.

[0011] The improvement of the present invention is that the physiological signal stability information includes EEG signal amplitude value, heart rate variability parameter, nerve activity reference value and RR interval change rate; the voice fluctuation characteristic data includes pitch change rate stability, speech rhythm offset rate, short-time pitch jitter index and the number of speech rate mutation points; the mental state fluctuation trend includes high-low frequency amplitude ratio, heart rate variability rhythm matching degree, voice pitch dynamic change range and nerve signal stability coefficient; the mental fatigue analysis information is specifically the short-time brain wave oscillation offset rate, Alpha wave and Theta wave alternation ratio, Beta wave transient decline amplitude and heart rate variability critical point determination value.

[0012] The improvement of the present invention is that the physiological signal analysis module includes:

[0013] Based on the intelligent wearable hat, the EEG signal amplitude calculation sub-module acquires the EEG signal, extracts the original amplitude data of the Delta, Theta, Alpha, and Beta frequency bands, performs discrete Fourier transform calculation on the amplitude data of each frequency band, obtains the spectral amplitude value, and performs normalization processing to obtain the normalized amplitude data of each frequency band;

[0014] The RR interval sequence calculation sub-module, based on the intelligent wearable hat, obtains the heart rate signal, extracts the time points of cardiac beat peaks, calculates the time differences between adjacent cardiac beat peaks, constructs the RR interval sequence, performs time-domain and frequency-domain feature analysis on the sequence, calculates the mean, standard deviation, and low-frequency / high-frequency power ratio of the RR interval, and obtains the time-frequency characteristic parameters of the RR interval;

[0015] The physiological signal stability evaluation sub-module calls the standardized amplitude data and the time-frequency characteristic parameters of the RR interval, and uses the formula:

[0016]

[0017] Calculates the physiological signal stability index and obtains the physiological signal stability information;

[0018] where S represents the physiological signal stability index, A i represents the standardized amplitude data of the i-th sampling point, represents the mean of the standardized amplitude data, R i represents the RR interval value of the i-th sampling point, represents the mean of the RR interval, and n represents the total number of sampling points.

[0019] The improvement of the present invention is that the speech data analysis module includes:

[0020] The pitch data extraction sub-module, based on the intelligent wearable hat, collects the speech signal, performs frame segmentation processing on the signal, extracts the fundamental frequency value of each frame of the speech signal, calls the short-time Fourier transform to calculate the spectral energy distribution, screens the fundamental frequency and harmonic components, and obtains the pitch data;

[0021] The speech rate change calculation sub-module, based on the pitch data, calculates the short-time energy change rate of the speech signal, detects the instantaneous speech rate between speech segments, calculates the speech rate change amplitude per unit time, extracts the speech rate mutation points, and calculates the distribution density to obtain the speech rate change trend;

[0022] The speech fluctuation feature calculation sub-module calls the speech rate change trend, calculates the pitch change rate, and performs a moving average operation on the pitch data, using the formula:

[0023]

[0024] Calculates the pitch curve fluctuation value to obtain the speech fluctuation feature data;

[0025] where D represents the pitch curve fluctuation value, F j represents the pitch data of the j-th sampling point, F j+1 represents the pitch data of the adjacent sampling point, T Nrepresents the timestamp of the last sampling point, T1 represents the timestamp of the first sampling point, and N represents the total number of sampling points.

[0026] The present invention is improved in that the mental fluctuation evaluation module includes:

[0027] The amplitude ratio calculation sub-module calls the physiological signal stability information, calculates the sum of the amplitude values of the low-frequency signal and the high-frequency signal, and calculates the ratio between the two to obtain the low-high frequency amplitude ratio;

[0028] The mental state calculation sub-module calculates the degree of fluctuation of the heart rate variability cycle within the time window based on the low-high frequency amplitude ratio and the voice fluctuation characteristic data, and calculates the mental state fluctuation amplitude in combination with the voice fluctuation data, using the formula:

[0029]

[0030] Performs an operation to obtain the mental state fluctuation index and obtains the degree of mental state fluctuation;

[0031] where M represents the mental state fluctuation index, U z represents the low-frequency amplitude value at the z-th moment, O z represents the high-frequency amplitude value at the z-th moment, W z represents the weight factor corresponding to the heart rate variability cycle, Z represents the total number of moments within the time window, and σ D represents the standard deviation of the tone curve fluctuation value;

[0032] The fluctuation trend acquisition sub-module calculates the change trend of the time series data based on the degree of mental state fluctuation and the mental state fluctuation values within a period of time, and obtains the mental state fluctuation trend.

[0033] The present invention is improved in that the fatigue boundary determination module includes:

[0034] The brain wave change calculation sub-module calls the mental state fluctuation trend, extracts the real-time frequency change rates of the Beta wave and the Theta wave, calculates the change amount of the fluctuation amplitude within a short-time window, and obtains the brain wave frequency change rate;

[0035] The amplitude offset determination sub-module calculates the alternating rate of the Alpha wave and the Theta wave based on the brain wave frequency change rate, and calculates the offset value of the amplitude change in combination with the resting state reference amplitude to judge the stability of the brain wave oscillation and obtain the amplitude offset index;

[0036] The mental fatigue calculation sub-module based on the amplitude offset index, in combination with the heart rate variability cycle, uses the formula:

[0037]

[0038] Calculate the mental fatigue index, compare it with the mental fatigue standard, determine whether it is at the mental fatigue boundary, and obtain the mental fatigue analysis information;

[0039] Among them, F s represents the mental fatigue index, Al a represents the Alpha wave amplitude at the a-th moment, Th a represents the Theta wave amplitude at the a-th moment, Be a represents the Beta wave amplitude at the a-th moment, N F represents the number of sampling points, σ P represents the standard deviation of the heart rate variability cycle.

[0040] The improvement of the present invention is that the system further includes:

[0041] The mental state analysis module based on the mental fatigue analysis information and the mental state fluctuation trend, screens the short-term high-fluctuation state, combines the neural amplitude oscillation ratio, evaluates the mental load level, calculates the mental health state score, and obtains the real-time mental state monitoring information;

[0042] The real-time mental state monitoring information includes the long-term mental state stability score, mental load index, short-term mental fluctuation classification, and neural amplitude consistency measure.

[0043] The improvement of the present invention is that the mental state analysis module includes:

[0044] The high-fluctuation state screening sub-module calls the mental fatigue analysis information and the mental state fluctuation trend, screens the intervals where the mental state fluctuation amplitude within a short-term window is higher than the set threshold, and records the duration of the fluctuation state to obtain the high-fluctuation state data;

[0045] The mental load calculation sub-module based on the high-fluctuation state data, calculates the neural amplitude oscillation ratio, and combines the heart rate variability data, using the formula:

[0046]

[0047] Calculate to obtain the mental load level;

[0048] Among them, L represents the mental load level, V b represents the neural amplitude oscillation ratio at the b-th moment, T b represents the duration of the high-fluctuation state at the b-th moment, τ represents the time decay coefficient, N L represents the number of sampling points for mental load calculation, B P represents the standard deviation of the heart rate variability data, and e is the base of the natural logarithm;

[0049] The mental health score calculation sub-module calculates the mental health status score based on the mental load degree, the mental load duration, and in combination with the individual mental state, to obtain the real-time monitoring information of the mental state.

[0050] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0051] In the present invention, by extracting key physiological and voice signal features, it is possible to more accurately judge the activity stability of the individual's neural signals and the autonomic nervous system, effectively evaluate the stability of physiological signals. By calculating the mental state fluctuation trend and combining physiological and voice data, it is possible to more comprehensively capture the real-time changes of the individual's emotional fluctuations and mental load, making the monitoring of the mental state more comprehensive and accurate. Based on the changes in physiological and voice signals, real-time assessment of mental fatigue is carried out, providing a more objective basis for fatigue determination, enhancing the accuracy of mental health state assessment, and providing more accurate mental health state information for the individual. Description of the Drawings

[0052] Figure 1 is the system flow chart of the present invention;

[0053] Figure 2 is the flow chart of the physiological signal analysis module of the present invention;

[0054] Figure 3 is the flow chart of the voice data analysis module of the present invention;

[0055] Figure 4 is the flow chart of the mental fluctuation assessment module of the present invention;

[0056] Figure 5 is the flow chart of the fatigue boundary determination module of the present invention;

[0057] Figure 6 is the flow chart of the mental state analysis module of the present invention. Detailed Embodiments

[0058] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, in the description of the present invention, the meaning of "a plurality of" is two or more unless otherwise specifically defined.

[0060] Please refer to Figure 1 , the present invention provides a technical solution: a real-time mental state monitoring system based on a smart wearable device, the system includes:

[0061] The physiological signal analysis module is based on a smart wearable hat, acquires electroencephalogram signal and heart rate signal data, extracts amplitude data of Delta, Theta, Alpha, and Beta frequency bands, calculates the RR interval sequence and the electromyogram activity frequency, judges the stability of nerve signals and autonomic nervous system activities, and obtains physiological signal stability information;

[0062] The voice data analysis module is based on a smart wearable hat, collects voice signals, extracts pitch data, calculates the speech rate mutation value, records the speech rate change trend within a unit time, calculates the pitch change rate, and obtains the mean deviation of the pitch curve to obtain voice fluctuation characteristic data;

[0063] The mental fluctuation evaluation module is based on the physiological signal stability information and the voice fluctuation characteristic data, calculates the amplitude oscillation ratio of low frequency (Delta + Theta) to high frequency (Alpha + Beta), combines the heart rate variability cycle and the voice fluctuation characteristic, calculates the mental state fluctuation degree, and obtains the mental state fluctuation trend;

[0064] The fatigue boundary determination module is based on the mental state fluctuation trend, extracts the real-time frequency change rate of high-frequency Beta waves and low-frequency Theta waves, calculates the fluctuation amplitude offset value within a short-time window, judges the stability of brain wave oscillation, calculates the Alpha wave and Theta wave alternation rate, identifies whether the amplitude change deviates from the resting state reference value, and combines the heart rate variability cycle to calculate the mental fatigue index to obtain mental fatigue analysis information;

[0065] The mental state analysis module is based on the mental fatigue analysis information and the mental state fluctuation trend, screens short-time high-fluctuation states, combines the nerve amplitude oscillation ratio, evaluates the mental load degree, calculates the mental health state score, and obtains the real-time mental state monitoring information.

[0066] The physiological signal stability information includes electroencephalogram signal amplitude values, heart rate variability parameters, neural activity baseline values, and RR interval change rates. The voice fluctuation characteristic data includes pitch change rate stability, speech rhythm deviation rate, short-term pitch jitter index, and number of speech speed mutation points. The mental state fluctuation trend includes high-frequency and low-frequency amplitude ratios, heart rate variability rhythm matching degree, voice pitch dynamic change range, and neural signal stability coefficient. The mental fatigue analysis information specifically includes short-term brain wave oscillation deviation rate, Alpha wave and Theta wave alternation ratio, Beta wave transient decline amplitude, and heart rate variability critical point determination value. The real-time mental state monitoring information includes long-term mental state stability score, mental load index, short-term mental fluctuation classification, and neural amplitude consistency measure.

[0067] Please refer to Figure 2 , the physiological signal analysis module includes:

[0068] The electroencephalogram signal amplitude calculation sub-module, based on the intelligent wearable hat, acquires the electroencephalogram signal, extracts the original amplitude data of the Delta, Theta, Alpha, and Beta frequency bands, performs discrete Fourier transform calculation on the amplitude data of each frequency band, obtains the spectral amplitude value, and performs normalization processing to obtain the normalized amplitude data of each frequency band;

[0069] The intelligent wearable hat acquires the electroencephalogram signal through multiple electrodes contacting the scalp. The electrode arrangement positions are based on the international 10-20 system, such as Fp1, Fp2, Cz, etc. During signal acquisition, the sampling frequency is set to 512 Hz, and each electrode is connected through a high-impedance amplifier to ensure signal quality. The acquired original electroencephalogram signal contains components of multiple frequency bands. In the time domain, the signal changes with time. To extract the amplitude data of the Delta (0.5 - 4 Hz), Theta (4 - 8 Hz), Alpha (8 - 13 Hz), and Beta (13 - 30 Hz) frequency bands, first, a band-pass filter is applied to intercept the signals of each frequency band respectively. The filtered signal is segmented by a window function, and the length of each window is set to 1 second to ensure sufficient data points for subsequent processing. For the data within the window, the fast Fourier transform (FFT) is applied to convert it to the frequency domain, and the spectral amplitude values corresponding to each frequency band are calculated. In the FFT calculation, the number of data points within the window is set to 1024 points. During normalization, based on the maximum amplitude value of each frequency band, the relative amplitude is calculated, that is to ensure the comparability of the amplitude characteristics of different frequency bands, where A max is set according to the statistical results of electroencephalogram signal experimental data. In general healthy individuals, the amplitude upper limits of different frequency bands are different. For example, the amplitude upper limit of the Delta frequency band usually does not exceed 50 μV, the Theta frequency band does not exceed 30 μV, the Alpha frequency band is in the range of 10 - 20 μV, and the Beta frequency band generally does not exceed 15 μV. Therefore, A maxThe specific value of is set according to this statistical range. If the maximum amplitude of the Delta frequency band in a certain test is measured to be 12 μV, then after normalization, the standardized amplitude value range of this frequency band is between [0, 1]. The same calculation method is applied to other frequency bands. After completing the standardization process, the standardized amplitude data of each frequency band is obtained.

[0070] The RR interval sequence calculation sub-module is based on the intelligent wearable hat to obtain the heart rate signal, extract the time points of the heart beat peaks, calculate the time difference between adjacent heart beat peaks, construct the RR interval sequence, perform time-domain and frequency-domain feature analysis on the sequence, calculate the mean, standard deviation, and low-frequency / high-frequency power ratio of the RR interval, and obtain the time-frequency characteristic parameters of the RR interval.

[0071] When the intelligent wearable hat collects the heart rate signal, it uses a photoplethysmogram (PPG) sensor or an electrocardiogram (ECG) to detect the time points of the heart beat peaks. The sampling rate of the PPG signal is set to 256 Hz, and the sampling rate of the ECG signal is set to 512 Hz. During the signal processing, first, the wavelet transform method is used to remove noise, and the adaptive threshold method is applied to detect the peaks. After detecting the peak points, using the peak time point sequence T i Calculate the time difference between adjacent peaks, that is, the RR interval R i = T i+1 - T i , construct the RR interval sequence. The time-domain analysis includes calculating the mean and the standard deviation σ R of the RR interval. For example, in a certain test sample, the measured RR interval sequence is [0.8, 0.85, 0.78, 0.82, 0.79] seconds, then the mean is calculated as The standard deviation is calculated as During the calculation process, the setting benchmark of the RR interval standard deviation σ R is the normal heart rate variability (HRV) range of healthy individuals. Usually, the HRV standard deviation of healthy adults is in the range of 0.02 - 0.05 seconds. If the calculation result of σ R exceeds 0.05 seconds, it may indicate large heart rate fluctuations. If it is lower than 0.02 seconds, it may indicate that the heart rate is too stable. The standard deviation calculation is used to further analyze the heart rate fluctuation situation. The frequency-domain analysis uses the power spectral density (PSD) to estimate the low-frequency (LF: 0.04 - 0.15 Hz) and high-frequency (HF: 0.15 - 0.4 Hz) components of the RR interval, integrates the power of LF and HF to obtain their respective energy values. For example, if the measured LF power is 0.18 Hz·ms2 and the HF power is 0.25 Hz·ms2, then the low-frequency / high-frequency power ratio (LF / HF) is calculated as 0.18 / 0.25 = 0.72, and finally the time-frequency characteristic parameters of the RR interval are obtained.

[0072] The physiological signal stability evaluation sub-module calls the standardized amplitude data and the RR interval time-frequency characteristic parameters, and uses the formula:

[0073]

[0074] Calculate the physiological signal stability index and obtain the physiological signal stability information;

[0075] Among them, S represents the physiological signal stability index, A i represents the standardized amplitude data at the i-th sampling point, represents the mean value of the standardized amplitude data, R i represents the RR interval value at the i-th sampling point, represents the mean value of the RR interval, and n represents the total number of sampling points;

[0076] When calculating the stability index S, the physiological signal stability evaluation sub-module calls the standardized amplitude data {A i} and the RR interval time-frequency characteristic parameters {R i}, and uses the formula for calculation. Set a sample data set as shown in Table 1. Set the standardized amplitude data {A i} to [0.4, 0.5, 0.42, 0.47, 0.45], and its mean value is calculated as Calculate the average value of the amplitude deviation as Then combine the RR interval data {R i} = [0.8, 0.85, 0.78, 0.82, 0.79], and calculate its sum of squared deviations

[0077]

[0078] Its square root value Finally, calculate the physiological signal stability index S = 0.0312 + 0.052 = 0.0832. Among them, the reference range of the stability index S is determined according to the normal physiological fluctuation range. Generally, a value between 0.05 and 0.1 belongs to the relatively stable range. A value lower than 0.05 indicates that the physiological signal is relatively stable, and a value higher than 0.1 indicates greater fluctuations.

[0079] Table 1: Example data of physiological signal parameters

[0080]

[0081] As shown in Table 1, the standardized amplitude data A i and the RR interval data R iFor calculating the stability index of physiological signals, and obtaining S = 0.0832 through the calculation process, this value characterizes the fluctuation degree of the current physiological signal.

[0082] Please refer to Figure 3 , the voice data analysis module includes:

[0083] The pitch data extraction sub-module, based on the intelligent wearable hat, collects voice signals, performs frame segmentation processing on the signals, extracts the fundamental frequency value of each frame of voice signal, calls the short-time Fourier transform to calculate the spectral energy distribution, screens the fundamental frequency and harmonic components, and obtains pitch data;

[0084] The intelligent wearable hat collects voice signals through a built-in microphone. The microphone sampling rate is set to 16 kHz. The signal is converted into a digital signal through an analog-to-digital conversion module. For subsequent processing convenience, the signal is subjected to frame segmentation processing. The length of each frame is set to 25 ms, and the adjacent frames overlap by 10 ms to ensure data continuity. Each frame contains multiple sampling points. For fundamental frequency value extraction with a single-frame signal, when calculating the fundamental frequency, first normalize the signal within the frame to eliminate the influence of amplitude differences on the calculation. Subsequently, calculate the short-time energy, and use the autocorrelation function (ACF) to detect the fundamental frequency period. For the autocorrelation sequence ACF(k) within the frame, search for the peak point, and convert the time interval T0 corresponding to the peak point into the fundamental frequency value, that is If the detected time interval corresponding to the autocorrelation peak is 0.008 seconds, then the fundamental frequency calculation is F0 = 125 Hz. After obtaining the fundamental frequency, apply the short-time Fourier transform (STFT) to calculate the spectral energy distribution of the frame. After the STFT transformation, calculate the local peak of the spectrum, screen out the fundamental frequency and harmonic components. The harmonic components are judged according to the integer multiple relationship. If the fundamental frequency is 125 Hz, then the harmonic components should appear at frequencies such as 250 Hz, 375 Hz, etc. Finally, obtain the pitch data. The threshold setting for fundamental frequency extraction is based on the vocal characteristics of the voice signal. Usually, the fundamental frequency fluctuates within the range of 85 Hz - 255 Hz. Therefore, set the threshold range [F min , F max = [85, 255] Hz to eliminate abnormal fundamental frequency signals. During the fundamental frequency extraction process, if the calculated fundamental frequency value exceeds this range, it is determined as invalid data and recalculated. For example, if the calculated fundamental frequency is 270 Hz, then because it exceeds the upper threshold of 255 Hz, this data needs to be eliminated.

[0085] The speech rate change calculation sub-module, based on the pitch data, calculates the short-time energy change rate of the voice signal, detects the instantaneous speech rate between speech segments, calculates the speech rate change amplitude per unit time, extracts the speech rate mutation points, and calculates the distribution density to obtain the speech rate change trend;

[0086] Based on the pitch data, calculate the short-time energy change rate of the speech signal. First, calculate the energy value of the sampled signal within a short-time window. The short-time energy is defined as where \(x(n)\) is the signal sample value. Calculate the energy change amount \(\Delta E(n)=E(n) - E(n - 1)\) between adjacent frames. The threshold for speech mutation point detection is set as \(T\) ΔE \(= 0.02E\) max where \(E\) max is the maximum short-time energy value of all frames in this section of speech. If the detected maximum short-time energy is \(E\) max \(= 0.5\), then the threshold for the mutation point is calculated as \(T\) ΔE \(= 0.02×0.5 = 0.01\). If the short-time energy change between adjacent frames is greater than \(0.01\), it is determined as a speech rate mutation point. The speech rate change amplitude per unit time is obtained by calculating the number of speech rate mutation points. For example, if 5 speech rate mutation points are detected within 2 seconds, then the speech rate change amplitude per unit time is \(5 / 2 = 2.5\) times per second. Subsequently, calculate the distribution density of the mutation points, which is the number of mutation points within the set time window divided by the window length. For example, if 25 mutation points are detected within a 10 - second window, then the distribution density is calculated as \(25 / 10 = 2.5\) times per second. Finally, obtain the speech rate change trend.

[0087] The speech fluctuation feature calculation sub - module calls the speech rate change trend, calculates the pitch change rate, and performs a moving average operation on the pitch data, using the formula:

[0088]

[0089] Calculate the pitch curve fluctuation value to obtain the speech fluctuation feature data;

[0090] where \(D\) represents the pitch curve fluctuation value, \(F\) j represents the pitch data of the \(j\) - th sampling point, \(F\) j+1 represents the pitch data of the adjacent sampling point, \(T\) N represents the time stamp of the last sampling point, \(T1\) represents the time stamp of the first sampling point, and \(N\) represents the total number of sampling points;

[0091] Based on the speech rate change trend, calculate the pitch change rate, and perform a moving average operation on the pitch data. Set the moving window length to 5 frames, calculate the mean value within each window to smooth the pitch curve, and apply the formula to the pitch data Calculate the pitch curve fluctuation value. Suppose the pitch data of the sampling points is \([120, 122, 118, 125, 121]\) Hz, and the sampling point time stamps are \([0.02, 0.04, 0.06, 0.08, 0.10]\) seconds respectively, as shown in Table 2. Then the fluctuation value is calculated as:

[0092]

[0093] Finally, the pitch curve fluctuation value is obtained, and the voice fluctuation feature data is acquired.

[0094] The reference value in the pitch change calculation is set according to the pitch change characteristics in the voice data. When calculating the pitch change rate, a reference time interval T needs to be set. base , usually a 0.5-second window is selected to calculate the average rate of pitch change. The pitch change rate is calculated for all frames within the window. If the calculated pitch change rate exceeds the set change reference value D base , it is determined as a severely changing area, and the reference value is set to D base = 1000 Hz² / s. If the calculated fluctuation value is 1062.5 Hz² / s as shown in the previous example, then this area is determined as a severely changing area.

[0095] Table 2: Example of voice signal feature data

[0096]

[0097] As shown in Table 2, the voice fundamental frequency data is used to calculate the pitch curve fluctuation value. Combining the calculation process, D = 1062.5 is obtained. This value is compared with the set change reference value D base = 1000 Hz² / s. Since 1062.5 is greater than 1000, it indicates that the voice fluctuation in this section is relatively severe.

[0098] Please refer to Figure 4 , the mental fluctuation evaluation module includes:

[0099] The amplitude ratio calculation sub-module calls the physiological signal stability information, calculates the sum of the amplitude values of the low-frequency signal (Delta + Theta) and the high-frequency signal (Alpha + Beta), and calculates the ratio between the two to obtain the low-high frequency amplitude ratio;

[0100] The amplitude ratio calculation sub-module calls the physiological signal stability information. First, it obtains the amplitude data of the low-frequency signal (Delta + Theta). The amplitude value of the low-frequency signal is calculated as where A Delta,i represents the amplitude of the Delta frequency band at the i-th sampling point, and A Theta,i represents the amplitude of the Theta frequency band at the i-th sampling point. At the same time, it obtains the amplitude data of the high-frequency signal (Alpha + Beta). The calculation method of the high-frequency amplitude value is the same as that of the low-frequency, namely When calculating the low-high frequency amplitude ratio, divide the sum of the low-frequency amplitude values by the sum of the high-frequency amplitude values. Set a certain test data. Set the low-frequency amplitude data as [3.2, 3.5, 3.8, 4.0, 4.3] μV, and the high-frequency amplitude data as [2.1, 2.4, 2.5,

[0101] 2.6 + 2.8 = 12.4 μV. Calculate the low-high frequency amplitude ratio as 18.8 / 12.4 = 1.516. Finally, obtain the low-high frequency amplitude ratio. Among them, the calculation of the low-frequency amplitude and the high-frequency amplitude involves the setting of the amplitude threshold. The threshold is set based on the distribution range of the normal EEG signal amplitude. In the state of conventional cognitive tasks, the amplitudes of the Delta and Theta frequency bands are usually between 1 - 10 μV, while the Alpha and Beta frequency bands are usually between 2 - 8 μV. Therefore, set the low-frequency amplitude threshold range at 1, 10 μV, and the high-frequency amplitude threshold range at 2, 8 μV. Signals outside this range are regarded as abnormal interference signals and are not included in the statistics when calculating the amplitude ratio.

[0102] The mental state calculation sub-module calculates the degree of fluctuation of the heart rate variability period within the time window based on the low-high frequency amplitude ratio and the voice fluctuation characteristic data, and calculates the mental state fluctuation amplitude in combination with the voice fluctuation data. Use the formula:

[0103]

[0104] Perform operations to obtain the mental state fluctuation index and get the degree of mental state fluctuation;

[0105] Among them, M represents the mental state fluctuation index, U z represents the low-frequency amplitude value at the z-th moment, O z represents the high-frequency amplitude value at the z-th moment, W z represents the weight factor corresponding to the heart rate variability period, Z represents the total number of moments within the time window, and σ D represents the standard deviation of the pitch curve fluctuation value;

[0106] The mental state calculation sub-module calculates the degree of fluctuation of the heart rate variability period within the time window based on the low-high frequency amplitude ratio and the voice fluctuation characteristic data. First, obtain the heart rate variability data HRV = [0.85, 0.88, 0.83, 0.90, 0.86] seconds, and calculate the standard deviation σ HRV of the heart rate variability. The standard deviation calculation formula where is the mean value of the heart rate variability. Calculate the mean value Calculate the standard deviation

[0107] Calculate the amplitude of mental state fluctuations in combination with voice fluctuation data, using the formula where U z is the low-frequency amplitude value at the z-th moment, O z is the high-frequency amplitude value at the z-th moment, W z is the weight factor corresponding to the heart rate variability cycle, Z is the total number of moments within the time window, and σ D is the standard deviation of the pitch curve fluctuation value. Set the low-frequency amplitude data within a certain time window as [3.2, 3.5, 3.8, 4.0, 4.3 μV], the high-frequency amplitude data as [2.1, 2.4, 2.5, 2.6, 2.8] μV, and the weight factors corresponding to heart rate variability as [0.9, 0.85, 0.88, 0.87, 0.86], as shown in Table 3. Set the standard deviation of the pitch curve fluctuation value to 0.052 and calculate the value of:

[0108]

[0109] Calculate Finally, obtain the mental state fluctuation index. Among them, the heart rate variability weight factor W z is set according to the individual heart rate variability characteristics. Under normal conditions, the HRV value usually ranges from 0.8 - 1.0 seconds. The weight factor of heart rate variability is set in the range of 0.8, 0.95. The larger the weight factor, the greater the influence of the HRV at that moment on the mental state. The HRV lower than 0.8 seconds corresponds to a faster heart rate, and the weight factor is set lower to reduce its influence in the calculation.

[0110] Table 3: Example of low- and high-frequency amplitude data

[0111] Sampling point <![CDATA[Low-frequency amplitude U z (μV)]]> <![CDATA[High-frequency amplitude O z (μV)]]> <![CDATA[Weight factor W z > 1 3.2 2.1 0.9 2 3.5 2.4 0.85 3 3.8 2.5 0.88 4 4.0 2.6 0.87 5 4.3 2.8 0.86

[0112] As shown in Table 3, the low-frequency amplitude, high-frequency amplitude, and weight factor data are used to calculate the mental state fluctuation index. Combining the calculation process, M = 1.374 is obtained, and this value characterizes the degree of mental state fluctuations.

[0113] The fluctuation trend acquisition sub-module calculates the change trend of the time series data based on the degree of mental state fluctuations and the mental state fluctuation values within a period of time, and obtains the mental state fluctuation trend;

[0114] The fluctuation trend acquisition sub-module calculates the change trend of the time series data based on the degree of mental state fluctuations and the mental state fluctuation values within a period of time, and obtains the mental state fluctuation trend. Set the time window length to 60 seconds, collect the mental state fluctuation index per second to form time series data, set the test data as [1.2, 1.3, 1.25, 1.4, 1.35], and calculate the change rate ΔM = M of the time seriest+1 -M t , the change rate data [0.1, -0.05, 0.15, -0.05] is obtained, and the trend value is calculated Finally, the mental state fluctuation trend is obtained. Among them, the setting of the time window length is based on the typical cycle of mental state changes. In cognitive tasks and emotional fluctuations, the stability of short-term mental state is usually more obvious within 30 - 90 seconds. Therefore, the time window is set at 60 seconds to take into account both short-term fluctuations and long-term change trends. If the standard deviation of the mental state fluctuation value is greater than 0.1, it indicates that the mental state fluctuates violently, and the time window needs to be adjusted for re-evaluation.

[0115] Please refer to Figure 5 , the fatigue boundary determination module includes:

[0116] The brain wave change calculation sub-module calls the mental state fluctuation trend, extracts the real-time frequency change rates of Beta wave and Theta wave, calculates the change amount of the fluctuation amplitude within a short-time window, and obtains the brain wave frequency change rate;

[0117] The brain wave change calculation sub-module calls the mental state fluctuation trend, first extracts the real-time frequency change rates of Beta wave and Theta wave. The calculation method of the real-time frequency change rate is to obtain the frequency data of Beta wave and Theta wave at consecutive moments, compare the frequency differences at adjacent moments, and define ΔF Beta,i =F Beta,i+1 -F Beta,i and ΔF Theta,i =F Theta,i+1 -F Theta,i , where F Beta,i and F Theta,i respectively represent the frequency values of Beta wave and Theta wave at the i-th moment. For example, if the data measured for the Beta wave frequency within consecutive moments is [15.8, 16.1, 16.4, 16.0, 15.7] Hz, and the data measured for the Theta wave frequency is [6.2, 6.4, 6.5, 6.3, 6.1] Hz, then the change rate ΔF Beta is [0.3, 0.3, -0.4, -0.3] Hz, ΔF ThetaIs [0.2, 0.1, -0.2, -0.2] Hz. Calculate the change amount of the fluctuation amplitude within a short-time window. Define the window length as 5 seconds. Calculate the total change amplitude ∑|ΔF| within the window and take the average value. For example, if the total change amplitude of the Beta wave is 0.3 + 0.3 + 0.4 + 0.3 = 1.3 Hz, then the change amount of the fluctuation amplitude within the short-time window is 1.3 / 5 = 0.26 Hz / s, and finally obtain the brain wave frequency change rate. Among them, the reference value of the short-time window length is set to 5 seconds. The basis for setting this value lies in the physiological characteristics of the EEG brain electrical signal. In the brain cognitive activity, the short-time changes of the Beta wave and Theta wave generally occur within the time scale of 2 - 10 seconds. Therefore, take the intermediate value of 5 seconds to balance sufficient time resolution and data integrity. If the short-time window is less than 3 seconds, it may lead to a significant impact of short-time noise. If it is greater than 7 seconds, it may affect the real-time responsiveness. This reference value changes with the task type. For example, when performing a highly concentrated task, the window length can be shortened to 3 seconds, and it can be extended to 7 seconds in the resting state.

[0118] Based on the brain wave frequency change rate, the amplitude offset determination sub-module calculates the alternation rate between the Alpha wave and the Theta wave, and combines the resting state reference amplitude to calculate the offset value of the amplitude change, determines the stability of the brain wave oscillation, and obtains the amplitude offset index;

[0119] Based on the brain wave frequency change rate, the amplitude offset determination sub-module calculates the alternation rate between the Alpha wave and the Theta wave, and combines the resting state reference amplitude to calculate the offset value of the amplitude change. First, obtain the continuous amplitude data of the Alpha wave and the Theta wave. The calculated alternation rate is defined as the number of amplitude changes between the Alpha wave and the Theta wave per unit time, that is, count the number of conversions of the amplitude high and low relationship between the two per unit time. For example, within a certain time window, the measured data of the Alpha wave amplitude is [4.1, 4.5, 4.3, 4.0, 4.2] μV, and the measured data of the Theta wave amplitude is [3.2, 3.0, 3.5, 3.8, 3.4] μV, as shown in Table 4. Then calculate whether the Alpha wave amplitude is higher than the Theta wave at each moment, and obtain the sequence [1, 1, 0, 0, 1]. The alternation number is 2 times. Calculate the alternation rate within a 5-second time window as 2 / 5 = 0.4 times / second. Combine the resting state reference amplitude to calculate the offset value of the amplitude change. Set the resting state reference Alpha amplitude to 4.0 μV and the Theta amplitude to 3.2 μV. Calculate the amplitude offset as defined as Where A base Is the resting state reference amplitude. For example, the amplitude offset calculation of the Alpha wave Finally, the amplitude offset index is obtained. Among them, the setting of the resting-state reference amplitude is based on the individual physiological parameters of the measured individual. The typical resting-state amplitude of Alpha waves is in the range of 3.5 - 5.0 μV, and the amplitude of Theta waves is in the range of 2.5 - 4.0 μV. The setting of this reference value is affected by the individual's daily EEG records. If the average Alpha amplitude in the individual's resting state is 4.0 μV, this is used as the reference. If the Alpha wave amplitude drops to 3.5 μV in the individual's resting state at night, the individual reference can be adjusted to 3.5 μV.

[0120] Table 4: Example of brain wave amplitude data

[0121] Sampling point Alpha wave amplitude (μV) Theta wave amplitude (μV) Beta wave amplitude (μV) 1 4.1 3.2 15.8 2 4.5 3.0 16.1 3 4.3 3.5 16.4 4 4.0 3.8 16.0 5 4.2 3.4 15.7

[0122] As shown in Table 4, the amplitude data of Alpha waves, Theta waves, and Beta waves are used to calculate the mental fatigue index. Combining the calculation process, we get F s = 0.0806, and this value characterizes the degree of mental fatigue

[0123] The mental fatigue calculation sub-module is based on the amplitude offset index, combines the heart rate variability cycle, and uses the formula:

[0124]

[0125] Performs operations to obtain the mental fatigue index, compares it with the mental fatigue index standard, determines whether it is at the mental fatigue boundary, and obtains the mental fatigue analysis information;

[0126] Among them, F s represents the mental fatigue index, Al a represents the Alpha wave amplitude at the a-th moment, Th a represents the Theta wave amplitude at the a-th moment, Be a represents the Beta wave amplitude at the a-th moment, N F represents the number of sampling points, and σ P represents the standard deviation of the heart rate variability cycle.

[0127] The mental fatigue calculation sub-module is based on the amplitude offset index, combines the heart rate variability cycle, and uses the formula

[0128] Performs operations to obtain the mental fatigue index. First, obtain the amplitude data of Alpha waves, Theta waves, and Beta waves at consecutive moments, and set N F= 5, the Alpha wave amplitude data of the sampling points are [4.1, 4.5, 4.3, 4.0, 4.2] μV, the Theta wave amplitude data are [3.2, 3.0, 3.5, 3.8, 3.4] μV, and the Beta wave amplitude data are [15.8, 16.1, 16.4, 16.0, 15.7] μV. Calculate the fatigue component of each sampling point The specific calculation is as follows:

[0129]

[0130] Calculate the average value Set the standard deviation σ of the heart rate variability cycle P = 0.028, and finally calculate the fatigue index F s = 0.0526 + 0.028 = 0.0806, and finally obtain the mental fatigue analysis information. Among them, the standard deviation σ of the heart rate variability cycle P is set based on the individual's daily HRV range. The typical HRV standard deviation is between 0.02 - 0.06 seconds. This value is related to the individual's activity state. For example, during exercise, σ P may increase to more than 0.05 seconds, and during deep sleep, σ P is usually less than 0.03 seconds. In this case, σ P = 0.028, based on the average HRV standard deviation measurement value of the individual at rest. If this value is higher than 0.04 seconds, it may indicate enhanced cardiac autonomic nerve regulation. If it is lower than 0.02 seconds, it may indicate increased fatigue or stress. This value can be dynamically adjusted after long-term measurement.

[0131] Please refer to Figure 6 , the mental state analysis module includes:

[0132] The high-fluctuation state screening sub-module calls the mental fatigue analysis information and the mental state fluctuation trend, screens the intervals where the mental state fluctuation amplitude within a short time window is higher than the set threshold, records the duration of the fluctuation state, and obtains the high-fluctuation state data;

[0133] The high-fluctuation state screening sub-module calls the mental fatigue analysis information and the mental state fluctuation trend, screens the intervals where the mental state fluctuation amplitude within a short time window is higher than the set threshold, records the duration of the fluctuation state, and obtains the high-fluctuation state data. First, obtain the mental state fluctuation data within the short time window, calculate the change rate of the fluctuation amplitude, and define the change rate of the fluctuation amplitude as ΔM i = M i+1 - M i where M iDenote the mental state fluctuation index at the \(i\)-th moment. For example, if the measured mental state fluctuation data is \([1.1, 1.3, 1.5, 1.2, 1.6]\), calculate the change rates as \([0.2, 0.2, -0.3, 0.4]\), and set the threshold \(M\). th to be \(0.25\). This threshold is based on the standard deviation range of the mental state fluctuation index in different populations through statistical analysis. The standard deviation of the normal state fluctuation index is calculated as where is the mean value. The value of \(\sigma\) is stable between \(0.2 - 0.3\) among multiple experimental individuals. Therefore, the threshold is set to \(0.25\) to ensure that the selected high-fluctuation states have significant characteristics. When \(|\Delta M i |>0.25\), it is determined as a high-fluctuation state. Screen the time periods that meet the conditions. For example, \(|\Delta M_3| = 0.3\) and \(|\Delta M_4| = 0.4\) meet the threshold. Calculate the duration of the high-fluctuation state. Define the duration as the time length that meets the threshold conditions. For example, if the time interval between two times of meeting the high-fluctuation condition is \(3\) seconds, then the duration of the high-fluctuation state is \(3\) seconds. Finally, obtain the high-fluctuation state data.

[0134] Based on the high-fluctuation state data, the mental load calculation sub-module calculates the neural amplitude oscillation ratio and combines it with the heart rate variability data, using the formula:

[0135]

[0136] to calculate the mental load degree through operation and obtain the mental load level;

[0137] where \(L\) represents the mental load degree, \(V b represents the neural amplitude oscillation ratio at the \(b\)-th moment, \(T b represents the duration of the high-fluctuation state at the \(b\)-th moment, \(\tau\) represents the time decay coefficient, \(N L represents the number of sampling points for mental load calculation, \(B P represents the standard deviation of the heart rate variability data, and \(e\) is the base of the natural logarithm;

[0138] Based on the high-fluctuation state data, the mental load calculation sub-module calculates the neural amplitude oscillation ratio and combines it with the heart rate variability data, using the formula to calculate the mental load degree through operation. First, obtain the neural amplitude oscillation ratio \(V b and the duration of the high-fluctuation state \(T b, set the sample data, the neural amplitude oscillation ratio is [0.8, 0.7, 0.9, 0.85, 0.75], the duration of the high-fluctuation state is [3, 4, 2, 5, 3] seconds, as shown in Table 5. Set the time decay coefficient τ = 2, which is calculated based on the recovery rate of short-term neural activity. Experimental measurements show that the neural activity intensity can decay to 37% (1 / e) of the initial value within 2 seconds. Therefore, the decay time constant is selected as 2 seconds to conform to the characteristics of physiological signal changes, and calculate the exponential decay term value:

[0139]

[0140] Calculate the load component at each moment:

[0141] 0.8 × 0.223 = 0.178;

[0142] 0.7 × 0.135 = 0.095;

[0143] 0.9 × 0.368 = 0.331;

[0144] 0.85 × 0.082 = 0.07;

[0145] 0.75 × 0.223 = 0.167;

[0146] Calculate the average value:

[0147]

[0148] Set the standard deviation B of the heart rate variability data P = 0.028. This standard deviation is calculated by long-term measurement of the heart rate variability data of individuals in different mental states. In the resting state, the standard deviation of heart rate variability is usually between 0.02 - 0.03. Therefore, 0.028 is selected as the reference value. Finally, calculate the mental load degree L = 0.1682 + 0.028 = 0.1962, and finally obtain the mental load level.

[0149] Table 5: High-fluctuation state and mental load calculation data

[0150]

[0151] As shown in Table 5, the neural amplitude oscillation ratio, the duration of the high-fluctuation state, and the exponential decay term data are used to calculate the mental load degree. The exponential decay term is calculated by the set time decay coefficient τ, and the value range conforms to the neural activity recovery characteristics. The standard deviation B of heart rate variability P is selected based on statistical analysis to ensure that its data range is consistent with the actual measurement data. Combining the calculation process, L = 0.1962 is obtained.

[0152] Based on the degree of mental load, according to the duration of mental load, and in combination with the individual mental state, the mental health score calculation sub-module calculates the mental health state score to obtain real-time monitoring information of the mental state;

[0153] Based on the degree of mental load, according to the duration of mental load, and in combination with the individual mental state, the mental health score calculation sub-module calculates the mental health state score to obtain real-time monitoring information of the mental state. Set the mental load duration data as [10, 12, 15, 9, 11] seconds, and define the score calculation method as S = 100 - (10×L) - (5×T L ), where T L is the mean value of the mental load duration, and calculate the mean value Calculate the mental health score:

[0154] S = 100 - (10×0.1962) - (5×11.4) = 100 - 1.962 - 57 = 41.038;

[0155] Finally, the mental health state score is obtained.

[0156] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes and apply them to other fields. However, as long as it does not depart from the technical solution content of the present invention, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A real-time mental state monitoring system based on intelligent wearable devices, characterized in that, The system includes: The physiological signal analysis module, based on the intelligent wearable hat, acquires electroencephalogram (EEG) signal and heart rate signal data, calculates the RR interval sequence and the electromyogram activity frequency, judges the stability of neural signals and the activities of the autonomic nervous system, and obtains physiological signal stability information; The voice data analysis module, based on the intelligent wearable hat, collects voice signals, extracts pitch data, calculates the speech rate mutation value, records the speech rate change trend within a unit time, calculates the pitch change rate, and obtains the mean deviation of the pitch curve to obtain voice fluctuation characteristic data; The mental fluctuation evaluation module, based on the physiological signal stability information and the voice fluctuation characteristic data, calculates the amplitude oscillation ratio of low frequency to high frequency, combines the heart rate variability period and the voice fluctuation characteristics, calculates the mental state fluctuation degree, and obtains the mental state fluctuation trend; The fatigue boundary determination module, based on the mental state fluctuation trend, calculates the fluctuation amplitude offset value within a short-time window, judges the EEG oscillation stability, calculates the alternation rate of Alpha wave and Theta wave, identifies whether the amplitude change deviates from the resting state reference value, and combines the heart rate variability period to calculate the mental fatigue index and obtain mental fatigue analysis information.

2. The real-time mental state monitoring system based on a smart wearable device according to claim 1, wherein The physiological signal stability information includes the EEG signal amplitude value, heart rate variability parameters, neural activity reference value, and RR interval change rate. The voice fluctuation characteristic data includes pitch change rate stability, speech rhythm offset rate, short-time pitch jitter index, and number of speech rate mutation points. The mental state fluctuation trend includes the low-frequency / high-frequency amplitude ratio, heart rate variability rhythm matching degree, voice pitch dynamic change range, and neural signal stability coefficient. The mental fatigue analysis information is specifically the short-time EEG oscillation offset rate, Alpha wave / Theta wave alternation ratio, transient decrease amplitude of Beta wave, and heart rate variability critical point determination value.

3. The real-time mental state monitoring system based on the smart wearable device according to claim 1, characterized in that, The physiological signal analysis module includes: The EEG signal amplitude calculation sub-module, based on the intelligent wearable hat, acquires the EEG signal, extracts the original amplitude data of the Delta, Theta, Alpha, and Beta frequency bands, performs discrete Fourier transform calculation on the amplitude data of each frequency band, obtains the spectral amplitude value, and performs normalization processing to obtain the normalized amplitude data of each frequency band; The RR interval sequence calculation sub-module, based on the intelligent wearable hat, acquires the heart rate signal, extracts the time points of heart beat peak values, calculates the time difference between adjacent heart beat peak values, constructs the RR interval sequence, performs time domain and frequency domain feature analysis on the sequence, and calculates the mean, standard deviation, and low-frequency / high-frequency power ratio of the RR interval to obtain the RR interval time-frequency characteristic parameters; The physiological signal stability evaluation sub-module calls the normalized amplitude data and the RR interval time-frequency characteristic parameters and uses the formula: to calculate the physiological signal stability index and obtain physiological signal stability information; Among them, S represents the physiological signal stability index, A i represents the normalized amplitude data at the i-th sampling point, represents the mean of the normalized amplitude data, R i represents the RR interval value at the i-th sampling point, represents the mean of the RR intervals, and n represents the total number of sampling points.

4. The real-time mental state monitoring system based on the smart wearable device according to claim 1, characterized in that The voice data analysis module includes: The pitch data extraction sub-module, based on the intelligent wearable hat, collects voice signals, performs frame segmentation processing on the signals, extracts the fundamental frequency value of each frame of voice signal, calls the short-time Fourier transform to calculate the spectral energy distribution, screens the fundamental frequency and harmonic components, and obtains the pitch data; Based on the pitch data, the speech rate change calculation sub-module calculates the short-time energy change rate of the speech signal, detects the instantaneous speech rate between speech segments, calculates the speech rate change amplitude per unit time, extracts the speech rate mutation points, and calculates the distribution density to obtain the speech rate change trend; The speech fluctuation feature calculation sub-module calls the speech rate change trend, calculates the pitch change rate, and performs a moving average operation on the pitch data using the formula: Calculate the pitch curve fluctuation value to obtain the speech fluctuation feature data; Among them, D represents the pitch curve fluctuation value, F j represents the pitch data of the j-th sampling point, F j+1 represents the pitch data of adjacent sampling points, T N represents the timestamp of the last sampling point, T1 represents the timestamp of the first sampling point, and N represents the total number of sampling points.

5. The real-time mental state monitoring system based on a smart wearable device according to claim 1, wherein The mental fluctuation evaluation module includes: The amplitude ratio calculation sub-module calls the physiological signal stability information, calculates the sum of the amplitude values of the low-frequency signal and the high-frequency signal, and calculates the ratio between the two to obtain the low-high frequency amplitude ratio; Based on the low-high frequency amplitude ratio and the speech fluctuation feature data, the mental state calculation sub-module calculates the fluctuation degree of the heart rate variability period within the time window, and combines the speech fluctuation data to calculate the mental state fluctuation amplitude using the formula: Perform an operation to obtain the mental state fluctuation index and get the mental state fluctuation degree; Among them, M represents the mental state fluctuation index, U z represents the low-frequency amplitude value at the z-th moment, O z represents the high-frequency amplitude value at the z-th moment, W z represents the weight factor corresponding to the heart rate variability period, Z represents the total number of moments within the time window, σ D represents the standard deviation of the pitch curve fluctuation value; Based on the mental state fluctuation degree, the fluctuation trend acquisition sub-module calculates the change trend of the time series data according to the mental state fluctuation values within a period of time to obtain the mental state fluctuation trend.

6. The real-time mental state monitoring system based on the smart wearable device according to claim 1, characterized in that The fatigue boundary determination module includes: The brain wave change calculation sub-module calls the mental state fluctuation trend, extracts the real-time frequency change rates of the Beta wave and the Theta wave, calculates the change amount of the fluctuation amplitude within the short-time window, and obtains the brain wave frequency change rate; Based on the brain wave frequency change rate, the amplitude offset determination sub-module calculates the alternating rate of the Alpha wave and the Theta wave, and combines the resting state reference amplitude to calculate the offset value of the amplitude change to judge the stability of the brain wave oscillation and obtain the amplitude offset index; Based on the amplitude offset index, the mental fatigue calculation sub-module combines the heart rate variability period using the formula: Perform an operation to obtain the mental fatigue index, compare it with the mental fatigue criterion, judge whether it is at the mental fatigue boundary, and obtain the mental fatigue analysis information; Among them, F s represents the mental fatigue index, Al a represents the Alpha wave amplitude at the a-th moment, Th a represents the Theta wave amplitude at the a-th moment, Be a represents the Beta wave amplitude at the a-th moment, N F represents the number of sampling points, σ P represents the standard deviation of the heart rate variability period.

7. The real-time mental state monitoring system based on a smart wearable device according to claim 1, wherein The system further includes: Based on the mental fatigue analysis information and the mental state fluctuation trend, the mental state analysis module screens the short-time high-fluctuation states, combines the neural amplitude oscillation ratio, evaluates the mental load degree, calculates the mental health state score, and obtains the real-time mental state monitoring information; The real-time mental state monitoring information includes the long-term mental state stability score, the mental load index, the short-time mental fluctuation classification, and the neural amplitude consistency measure.

8. The real-time mental state monitoring system based on the smart wearable device according to claim 7, characterized in that, The mental state analysis module includes: The high-fluctuation state screening sub-module calls the mental fatigue analysis information and the mental state fluctuation trend, screens the intervals where the mental state fluctuation amplitude within the short-time window is higher than the set threshold, and records the duration of the fluctuation state to obtain the high-fluctuation state data; Based on the high-fluctuation state data, the mental load calculation sub-module calculates the neural amplitude oscillation ratio, and combines the heart rate variability data using the formula: Perform an operation to obtain the mental load degree and get the mental load level; Among them, L represents the mental load level, V b represents the neural amplitude oscillation ratio at the b-th moment, T b represents the duration of the high-fluctuation state at the b-th moment, τ represents the time decay coefficient, N L represents the number of sampling points for mental load calculation, B P represents the standard deviation of the heart rate variability data, and e is the base of the natural logarithm; The mental health score calculation sub-module calculates the mental health status score based on the mental load degree, the mental load duration, and the individual mental state, so as to obtain the real-time monitoring information of the mental state.

Images