Intelligent warning system for fatigue driving based on physiological signal monitoring technology

Abstract

The present invention belongs to the field of intelligent driving assistance technology, and discloses an intelligent early warning system for fatigue driving based on physiological signal monitoring technology. The system includes a physiological signal sensor module, a signal processing and analysis module, an early warning module, and a data storage and transmission module. The sensor module collects multimodal physiological signals such as EEG, ECG, eye movement, and skin electricity. The signal processing module preprocesses, extracts features, performs fusion analysis, and builds a personalized physiological model for the signal to accurately judge the fatigue state. The early warning module communicates with the intelligent driving system, coordinates the multi-dimensional early warning triggering timing, sends control instructions when fatigued, triggers sound, visual and vibration early warnings, and also performs time management. The data storage and transmission module stores local data and uploads it to the cloud. The system can effectively judge the driver's fatigue state and issue early warnings in time, and work with the intelligent driving system to ensure driving safety.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving assistance technology, and in particular to an intelligent early warning system for fatigue driving based on physiological signal monitoring technology. Background Art

[0002] Fatigue driving refers to the phenomenon that the driver becomes physically and mentally fatigued after driving continuously for a long time, which leads to a decline in driving skills. It usually manifests as drowsiness, lack of concentration, slow reaction, blurred vision, etc. Generally, continuous driving for more than a certain period of time (such as 4 hours) without adequate rest may be considered fatigue driving.

[0003] With the continuous increase in the number of cars and the rapid development of the transportation industry, fatigue driving has become one of the important causes of traffic accidents. In order to reduce the safety hazards caused by fatigue driving, researchers have been working hard to develop an effective intelligent warning system for fatigue driving. The information acquisition unit of the system collects driver information (including facial features, eye signals, head mobility, etc.) through various sensors and sends these signals to the electronic control unit. After receiving the signal, the electronic control unit performs calculation analysis to determine whether the driver is in a fatigue state. If it is judged to be a fatigue state, a signal will be sent to the warning display unit, and the warning display unit will issue warning prompts in various ways according to the information transmitted by the electronic control unit.

[0004] Although the above-mentioned early warning system uses advanced collection and analysis technology and can play an early warning role, it still has some shortcomings in practical application:

[0005] First, most of the current fatigue driving warning systems rely on a single physiological signal for judgment, or multiple physiological signals for independent judgment. However, a single signal is easily affected by external environmental factors and individual differences, which may lead to misjudgment. At the same time, there are inherent correlations and complementarities between different physiological signals, but the existing systems have not fully explored these characteristics and lack in-depth research on multimodal physiological signal fusion algorithms, which limits the accuracy and reliability of fatigue status judgment.

[0006] Secondly, traditional fatigue driving warning systems usually use universal judgment criteria, and are not personalized for different drivers' physiological characteristics such as age, gender, physical condition, and driving environment. Different drivers may show different fatigue symptoms, but the existing system cannot make accurate fatigue judgments based on these individual factors. Moreover, the system cannot be continuously updated and optimized over time during use, making it difficult to better adapt to the actual situation of the driver, resulting in insufficient pertinence and accuracy of the warning, and failing to provide the driver with a truly effective fatigue driving warning.

[0007] Finally, the existing warning system is not closely linked to the intelligent driving system. Often, only simple warning reminders are issued without coordinating with the intelligent driving system to perform operations such as vehicle speed control. When the driver is in a fatigued state, simple sound or visual warnings may not be able to effectively avoid accidents in a timely manner. In addition, after the warning is issued, the existing system does not adequately consider the restrictions on the driver's operations. When the driver is frightened in a fatigued state, he may make panic operations, and the current system cannot reasonably restrict the operation of the steering wheel, brakes, and accelerator during the warning period, which undoubtedly increases the risk of accidents. The linkage between the multi-dimensional warning and the intelligent driving system is not sufficient, and it is impossible to provide comprehensive safety protection for the driver. Summary of the invention

[0008] The main purpose of the present invention is to provide an intelligent early warning system for fatigue driving based on physiological signal monitoring technology, which can effectively solve the problems mentioned in the background technology.

[0009] To achieve the above object, the technical solution adopted by the present invention is:

[0010] The intelligent early warning system for fatigue driving based on physiological signal monitoring technology includes a physiological signal sensor module, a signal processing and analysis module, an early warning module, and a data storage and transmission module. Specifically:

[0011] The physiological signal sensor module includes: an electroencephalogram sensor for collecting the driver's electroencephalogram signal; an electrocardiogram sensor for monitoring the driver's heart rate changes; an eye movement sensor for detecting the driver's eye movement; and a skin electrode sensor for measuring the driver's skin electrode response.

[0012] The signal processing and analysis module includes a signal preprocessing unit, a feature extraction unit, a fusion analysis unit and a personalized physiological model construction unit; wherein the signal preprocessing unit filters, amplifies and denoises the physiological signals collected by the physiological signal sensor module; the feature extraction unit extracts key feature parameters that can reflect the driver's fatigue state from the preprocessed physiological signals; the fusion analysis unit uses a weighted fusion algorithm to comprehensively analyze the key feature parameters of different physiological signals and verify each other; the personalized physiological model construction unit uses the physiological signal data of the driver in a specific time period collected in the early stage and the driver's personal information to construct a personalized physiological model through a machine learning algorithm; and continuously optimizes and updates the model as new data is continuously input during use;

[0013] The warning module includes a communication and coordination submodule, a warning trigger and control submodule and a time management submodule, wherein the communication and coordination submodule establishes communication with the intelligent driving system of the driven vehicle, can send instructions to the intelligent driving system and receive vehicle status information fed back by the intelligent driving system; and coordinates the triggering timing of sound, visual and vibration warnings at the same time; the warning trigger and control submodule determines the driver's fatigue state according to the physiological model constructed by the signal processing and analysis module, and not only controls the triggering of sending control instructions to the intelligent driving system, starting the vehicle speed adjustment and operation restriction functions, but also controls the triggering of sound warnings, visual warnings and vibration warnings at the same time; the time management submodule controls the time process after the warning is issued, starts timing after the warning is issued, continuously executes the operation restriction within the preset time and maintains continuous warnings, and sends a release instruction to the intelligent driving system after the preset time ends to restore the normal operation state of the vehicle, while gradually weakening or turning off the sound, visual and vibration warnings;

[0014] The data storage and transmission module includes two parts: data storage and data transmission. The data storage stores the collected physiological signals and analysis results in the local memory; the data transmission transmits the physiological signals and analysis results to the cloud server through wireless communication technology.

[0015] Preferably, the EEG sensor is an elastic headband with built-in sensor electrodes, which is worn on the forehead and detects changes in electrical signals generated by brain neuron activity through contact between the electrodes and the scalp. The frequency and amplitude of these electrical signals can reflect the activity state of the brain; the ECG sensor and the skin electrical sensor are integrated on an elastic wristband, which is worn on the wrist and records changes in electrical signals generated each time the heart beats through electrodes, thereby monitoring the heart rate and changes in heart rate. When the driver is tired, the heart rate may fluctuate abnormally, such as faster or slower heart rate, reduced heart rate variability, etc.; and measure changes in skin resistance. When the driver is in different psychological states, the sweat gland secretion of the skin will change, resulting in changes in skin resistance. By detecting this resistance change, the driver's psychological pressure and tension can be reflected; the eye movement sensor is installed near the instrument panel directly in front of the driver or at the rearview mirror inside the car, and uses optical principles or infrared technology to detect the driver's eye position, movement trajectory and blinking frequency. When the driver is tired, the blinking frequency will increase, and the gaze time may become shorter or unstable.

[0016] Preferably, the specific processing steps of the signal preprocessing unit include:

[0017] ①Select Daubechies wavelet basis function;

[0018] ② Determine the number of decomposition layers based on the frequency range of the signal and the frequency components that need to be filtered; the more decomposition layers there are, the more finely the different frequency components of the signal can be analyzed.

[0019] ③ Perform multi-scale decomposition on the signal to obtain wavelet coefficients and extract physiological signal components in different frequency ranges; lower frequency scales may correspond to physiological signal changes related to the driver's long-term fatigue state, such as slowly changing low-frequency components of EEG signals or low-frequency parts of heart rate variability. Higher frequency scales may reflect the driver's stress response at a specific moment or physiological signal changes in a short fatigue moment, such as high-frequency components of eye movement signals or skin electrical signals.

[0020] ④ For the filtering operation, select the corresponding scale of the retained frequency components according to the filtering requirements, and reconstruct the filtered signal through inverse transformation.

[0021] ⑤ For denoising operation, soft threshold or hard threshold method is used to perform threshold processing on the wavelet coefficients, and then the denoised signal is reconstructed through inverse transform.

[0022] ⑥ For the amplification operation, determine the frequency scale to be amplified, amplify the corresponding scale coefficient by multiplying it by the amplification factor, and finally reconstruct the amplified signal by inverse transformation.

[0023] Preferably, the key feature parameters extracted by the feature extraction unit are divided into:

[0024] ① For EEG sensor signals, extract the power spectrum density, focusing on analyzing the power changes of α, β, θ, and δ waves in different frequency bands; these parameters can reflect the changes in brain activity with fatigue. The calculation of power spectrum density can be achieved through fast Fourier transform (FFT). After the EEG signal is framed, the FFT algorithm is used to convert the time domain signal to the frequency domain, so as to calculate the power spectrum density of different frequency bands.

[0025] ② For the ECG sensor signal, extract the heart rate variability index, including the standard deviation of adjacent heartbeat intervals (SDNN), and extract the average heart rate and heart rate change trend; the SDNN calculation formula is:

[0026]

[0027] in is the standard deviation, is the number of heartbeat intervals, For the heartbeat interval, The average heart rate can be calculated by simply counting the number of heartbeats in a certain period of time and averaging them.

[0028] ③ For the eye movement sensor signal, extract the blink frequency, extract the fixation time, including the average fixation time and the stability of the fixation time, and extract the saccade amplitude and speed; the blink frequency can be counted by detecting the changes in the closed state of the eyeball through the eye movement sensor. The fixation time, saccade amplitude and speed can be calculated by analyzing the eye movement trajectory data recorded by the eye movement sensor.

[0029] ④ For the galvanic skin sensor signal, extract the galvanic skin response amplitude and the galvanic skin response change rate. The galvanic skin response amplitude and change rate can be obtained by analyzing and calculating the resistance change data collected by the galvanic skin sensor.

[0030] Preferably, the weighted fusion algorithm adopted by the fusion analysis unit assigns different weights to each physiological signal according to its importance, and then fuses the weighted key feature parameters. First, the weight of each physiological signal is determined, and the weight value can be determined in advance through experiments or expert experience. Then, the key feature parameters of each physiological signal are multiplied by the corresponding weight to obtain the weighted key feature parameters. Finally, the weighted key feature parameters are summed to obtain the fused feature value, and the specific expression is:

[0031]

[0032] in is the fused eigenvalue, is the number of physiological signals, For the The weight of the physiological signal, For the characteristic parameters of a physiological signal.

[0033] Preferably, the method for constructing the personalized physiological model selects a support vector machine (SVM) as a machine learning algorithm, adopts a radial basis function (RBF) kernel function, uses training data to train the SVM model, solves the model parameters through a sequence minimum optimization algorithm, inputs the physiological signal data into the trained personalized physiological model, and judges the driver's fatigue state through a decision function, which is expressed as follows:

[0034]

[0035] in is the decision function value, used to determine the input feature vector The corresponding fatigue state, The input feature vector is composed of key feature parameters of multiple physiological signals. is the weight vector, It is a bias item; according to the fatigue status judgment result, early warning information is issued in time and corresponding intervention measures are taken.

[0036] The optimization method of the personalized physiological model includes cross-validation and regularization. The cross-validation randomly divides the data set into K subsets of equal size. The data set refers to the initial data set used to build and train the personalized physiological model. This data set usually contains physiological signal data and personal information collected by multiple drivers in a specific time period. Each time, one of the subsets is selected as the validation set, and the remaining K-1 subsets are used as the training set; model training and validation are repeated K times; K validation results are obtained, and the average value is taken as the final model performance evaluation indicator; the regularization determines the loss function with a regularization term:

[0037]

[0038] in is the total loss function, which is used to measure the performance of the model. is the original loss function, which is the difference between the model prediction value and the true value. is a regular term used to prevent the model from overfitting. is the regularization parameter, which is used to control the weight of the regularization term in the total loss function.

[0039] The updating method of the personalized physiological model adopts online learning. Online learning can update the model parameters as soon as new data appears, quickly adapt to data changes, and is suitable for scenarios that require rapid response. When new data arrives, the stochastic gradient descent (SGD) algorithm is used to update the model and calculate the gradient of the loss function to the model parameters. The gradient represents the direction and rate of change of the loss function at the current parameter. According to the formula Update the model parameters, where is the updated parameter, is the current parameter, is the learning rate, which controls the step size of parameter updates, The loss function is at the current parameter The gradient at represents the direction and rate of change of the loss function at that point. The new data refers to the changes in the driver's physiological signal data and personal information that are continuously collected over time after the model is built and trained. For example, after the model is put into use, the driver's physiological signals continue to be monitored, and the newly collected data can be used to update and optimize the personalized physiological model so that the model can adapt to changes in the driver's physiological state and personal situation. New data can help the model continuously learn and adjust to maintain the accuracy of judging the driver's fatigue state.

[0040] Preferably, the communication and coordination submodule selects Bluetooth, Wi-Fi or vehicle communication protocol (CAN bus, etc.) to establish a connection channel, continuously monitors the output results of the signal processing and analysis module, and prepares to send instructions to the intelligent driving system once it is determined that the driver is in a fatigue state; at the same time, it receives the vehicle status information fed back by the intelligent driving system, and coordinates the triggering timing of sound, visual and vibration warnings based on the fatigue state judgment and vehicle status information. The establishment of an efficient communication channel is to ensure that the warning module can interact with the intelligent driving system in a timely manner. When signs of fatigue driving are detected, instructions need to be sent to the intelligent driving system to adjust the vehicle speed and limit operations to ensure driving safety. Receiving vehicle status information allows the warning module to better understand the current driving situation, thereby more accurately coordinating the triggering timing of the warning. By reasonably coordinating the triggering timing of different warning methods, the warning can be made more effective while avoiding conflicts with the actions of the intelligent driving system.

[0041] Preferably, the early warning trigger and control submodule continuously receives the fatigue state judgment result from the signal processing and analysis module. When it is judged that the driver is in a fatigue state, a control instruction is immediately sent to the intelligent driving system to start the vehicle speed adjustment and operation restriction functions. At the same time, the sound warning, visual warning and vibration warning are triggered, and the intensity and combination of various warning methods are adjusted according to the different fatigue levels; wherein the sound warning is the in-car speaker in the cab, the visual warning is the warning icon or flashing light displayed on the dashboard, and the vibration warning is the vibration of the seat or steering wheel. When it is judged that the driver is fatigued, sending a control instruction to the intelligent driving system can directly intervene in the driving state of the vehicle and reduce the risk of accidents. The simultaneous triggering of multiple early warning methods can remind the driver from different sensory channels and improve his alertness to the fatigue state. The warning intensity and combination are adjusted according to the fatigue level because different degrees of fatigue require different degrees of stimulation to attract the driver's attention, ensuring that the driver can be effectively reminded in various fatigue situations.

[0042] Preferably, the time management submodule starts the timer immediately after the warning is triggered, and continuously monitors and ensures that the operation restrictions of the steering wheel, brake, and throttle remain in effect within the preset time; at the same time, the sound, visual, and vibration warning methods also continue to play a role, constantly reminding the driver; when the preset time is over, an instruction to lift the operation restrictions is sent to the intelligent driving system to restore the vehicle to normal operation, and gradually weaken or turn off the sound, visual, and vibration warnings. The purpose of setting time management is to force the driver to respond to fatigue within a certain period of time, while giving him a certain buffer time to recover. Continuous operation restrictions can prevent the driver from continuing to perform dangerous operations in a fatigued state. Lifting the operation restrictions after the preset time is considered to allow the driver to drive normally if he has regained consciousness or taken corresponding measures. Gradually weakening or turning off the warning can prevent the warning from continuing to interfere with the driver when it is not needed, so that the driving environment returns to normal.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] For multimodal physiological signal fusion, a new fusion algorithm is used to deeply analyze the inherent correlation and complementarity between different physiological signals. For example, when the EEG signal shows fatigue characteristics, the fatigue state is further confirmed by combining the change trend of the ECG signal to avoid misjudgment of a single signal. Through a large number of experiments and data analysis, the parameters of the fusion algorithm are optimized to adapt to the physiological characteristics and driving environment of different drivers, and to improve the accuracy and reliability of fatigue judgment.

[0045] In the early stage of system use, the system builds a personalized physiological model by monitoring and analyzing the driver's physiological signals over a period of time. This model takes into account factors such as the driver's age, gender, and physical condition, and can more accurately determine the driver's fatigue state. As the use time increases, the system continuously updates and optimizes the personalized physiological model to make it more suitable for the driver's actual situation and improve the pertinence and accuracy of fatigue driving warnings.

[0046] The warning module is linked to the intelligent driving system. While preparing to issue a warning reminder, it also coordinates with the intelligent driving system to control the vehicle speed. When the driver's fatigue is detected, the system will send instructions to the intelligent driving system according to the degree of fatigue and start the speed adjustment function to varying degrees. Within a certain period of time after the warning is issued, the operation of the steering wheel, brake, and accelerator will be restricted to prevent the driver from making panic operations when frightened. After the preset time is over, normal operation will be resumed. Through the linkage of multi-dimensional warnings (sound, vision, vibration) and the intelligent driving system, more comprehensive safety protection is provided for the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a schematic diagram of the system composition architecture flow of the present invention;

[0048] Figure 2 The signal processing and analysis module workflow diagram of the present invention;

[0049] Figure 3 It is the workflow diagram of the early warning module of the present invention. DETAILED DESCRIPTION

[0050] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0051] like Figure 1-Figure 3 As shown in the figure, the fatigue driving intelligent warning system based on physiological signal monitoring technology is mainly composed of a physiological signal sensor module, a signal processing and analysis module, a warning module, and a data storage and transmission module. The signal processing and analysis module first pre-processes the signal from the physiological signal sensor, including filtering, amplification, and denoising. Then, the key characteristic parameters of different physiological signals are extracted, such as the power spectrum density of the EEG signal. Then, these parameters are comprehensively analyzed by the weighted fusion algorithm. Finally, a personalized physiological model is constructed using the driver's specific data, and optimized and updated by cross-validation, regularization, and online learning. The communication and coordination submodule in the warning module establishes communication with the intelligent driving system, receives vehicle status information, and coordinates the timing of warning triggering. When judging the driver's fatigue, the warning trigger and control submodule sends control instructions to the intelligent driving system and triggers a multi-dimensional warning, and adjusts the warning intensity and combination according to the degree of fatigue. The time management submodule counts after the warning, continuously executes the operation restrictions and warnings within the preset time, and lifts the restrictions and weakens or turns off the warning after the end.

[0052] The following is a specific example of an intelligent early warning system for fatigue driving based on physiological signal monitoring technology:

[0053] 1. System installation and initialization

[0054] Wear an elastic headband made of EEG sensors on the driver's forehead to ensure that the electrodes are in good contact with the scalp, which can accurately detect changes in electrical signals generated by brain neuron activity. Wear an elastic wristband with integrated ECG sensors and skin electrical sensors on the driver's wrist, adjust it to be comfortable and the electrodes can stably record cardiac electrical signals and measure changes in skin resistance. Install the eye movement sensor near the dashboard directly in front of the driver or at the rearview mirror inside the car, and debug it to ensure that it can accurately detect the driver's eye position, movement trajectory and blinking frequency using optical principles or infrared technology.

[0055] After the system is started, it is initialized and each sensor begins to collect physiological signals and transmit them to the signal processing and analysis module.

[0056] 2. Signal Processing and Analysis

[0057] Signal preprocessing:

[0058] The signal preprocessing unit in the signal processing and analysis module first selects the Daubechies wavelet basis function. According to the previous test of the effect of different wavelet basis functions in physiological signal processing, it is determined that this basis function can better process various physiological signals in this system.

[0059] The number of decomposition layers is determined based on the frequency range of the signal and the frequency components that need to be filtered. For example, for EEG signals, considering that the frequency range is wide and some high-frequency noise and low-frequency interference need to be filtered out, the number of decomposition layers is determined to be 4 after analysis. This allows for a more detailed analysis of the different frequency components of the signal and the extraction of features related to fatigue status.

[0060] The signal is decomposed at multiple scales to obtain wavelet coefficients and extract physiological signal components in different frequency ranges. Taking the ECG signal as an example, the lower frequency scale may correspond to the low-frequency part of the heart rate variability related to the driver's long-term fatigue state, while the higher frequency scale may reflect the heart rate changes caused by the driver's stress response at a specific moment.

[0061] For filtering operations, the corresponding scale of the retained frequency components is selected according to the filtering requirements, and the filtered signal is reconstructed through inverse transformation. For example, for eye movement signals, in order to remove high-frequency interference caused by factors such as ambient light, the wavelet coefficients with lower frequency scales are selected to reconstruct the filtered eye movement signal.

[0062] For denoising, soft threshold or hard threshold method is used to perform threshold processing on wavelet coefficients, and then the denoised signal is reconstructed through inverse transformation. For example, for the galvanic skin sensor signal, according to the noise characteristics of the signal, a suitable threshold method is selected to remove noise interference and improve the quality of the signal.

[0063] For the amplification operation, the frequency scale to be amplified is determined, the corresponding scale coefficient is amplified by multiplying the amplification factor, and finally the amplified signal is reconstructed by inverse transformation. For example, for a specific frequency component in the EEG signal, such as the θ wave that is closely related to the fatigue state, its characteristics can be highlighted by amplifying the coefficient of the frequency scale for subsequent analysis.

[0064] Feature extraction:

[0065] For EEG sensor signals, the power spectrum density is extracted, focusing on analyzing the power changes of α, β, θ, and δ waves in different frequency bands. The EEG signal is converted from the time domain to the frequency domain through the fast Fourier transform (FFT), and the power spectrum density of different frequency bands is calculated. For example, when the system detects an increase in the power of the θ wave and a decrease in the power of the α wave, it is preliminarily judged that the driver may be in a state of fatigue.

[0066] For the ECG sensor signal, extract the heart rate variability index, including the standard deviation of adjacent heartbeat intervals (SDNN), and extract the average heart rate and heart rate change trend. Calculate the SDNN according to the formula, and calculate the average heart rate by counting the number of heartbeats in a certain period of time. For example, after the driver has been driving for a long time, the system detects that the heart rate variability decreases and the average heart rate increases, indicating that the driver may be fatigued.

[0067] For the eye movement sensor signal, the blink frequency, fixation time (including average fixation time and stability of fixation time), scan amplitude and speed are extracted. The blink frequency is counted by detecting the changes in the closed state of the eyeball through the eye movement sensor, and the fixation time, scan amplitude and speed are calculated by analyzing the eye movement trajectory data. For example, when the system detects that the blink frequency increases, the fixation time becomes shorter, and the scan amplitude and speed decrease, it indicates that the driver may be fatigued.

[0068] For the galvanic skin sensor signal, the galvanic skin response amplitude and the galvanic skin response change rate are extracted. These parameters are obtained by analyzing and calculating the resistance change data collected by the galvanic skin sensor. For example, when the driver is in a state of tension or fatigue, the galvanic skin response amplitude increases and the change rate accelerates.

[0069] Fusion analysis:

[0070] A weighted fusion algorithm is used to assign different weights to each physiological signal according to its importance. For example, through previous experiments and expert experience, the weight of the EEG signal is determined to be 0.4, the weight of the ECG signal is 0.3, the weight of the eye movement signal is 0.2, and the weight of the skin electrical signal is 0.1. Then the weighted key feature parameters are fused, and the fused feature value is calculated according to the fusion formula. When the fused feature value exceeds the preset threshold, it is judged that the driver is in a fatigue state.

[0071] Personalized physiological model construction and optimization update:

[0072] Using the physiological signal data of the driver in a specific time period collected in the early stage and the driver's personal information, the support vector machine (SVM) is used as the machine learning algorithm, the radial basis function (RBF) kernel function is used, the SVM model is trained using the training data, and the model parameters are solved by the sequence minimum optimization algorithm to build a personalized physiological model. The physiological signal data is input into the trained personalized physiological model, and the driver's fatigue state is judged through the decision function.

[0073] The optimization methods of personalized physiological models include cross-validation and regularization. Cross-validation randomly divides the data set into K subsets of equal size, such as K=5. Each time, one of the subsets is selected as the validation set, and the remaining 4 subsets are selected as the training set. Model training and validation are repeated 5 times to obtain 5 validation results, and the average value is taken as the final model performance evaluation indicator. Regularization determines the loss function with regularization terms. By adjusting the regularization parameters, the weight of the regularization term in the total loss function is controlled to prevent the model from overfitting.

[0074] The updating method of the personalized physiological model adopts online learning. When new data arrives, the model is updated using the stochastic gradient descent (SGD) algorithm. For example, after the model is put into use, the driver's physiological signals continue to be monitored, and the newly collected data can be used to update and optimize the personalized physiological model so that the model can adapt to changes in the driver's physiological state and personal situation.

[0075] 3. Early Warning

[0076] Communications and Coordination:

[0077] The communication and coordination submodule in the early warning module selects Bluetooth to establish an efficient communication channel with the intelligent driving system. When the system determines that the driver may be in a fatigued state, it prepares to send instructions to the intelligent driving system. At the same time, it receives the vehicle status information fed back by the intelligent driving system, and coordinates the triggering timing of sound, visual and vibration warnings based on the fatigue status judgment and vehicle status information. For example, when the vehicle is driving on a mountainous road with many bends and the driver is in a fatigued state, the communication and coordination submodule coordinates the warning triggering timing based on information such as the vehicle's driving speed and steering angle, and gives priority to triggering visual warnings and vibration warnings to attract the driver's attention.

[0078] Early warning trigger and control:

[0079] When it is determined that the driver is in a fatigue state, the warning trigger and control submodule immediately sends a control command to the intelligent driving system to start the vehicle speed adjustment and operation restriction functions. At the same time, the sound warning, visual warning and vibration warning are triggered, and the intensity and combination of various warning methods are adjusted according to the degree of fatigue. For example, when the fatigue level is light, the sound warning can be a soft prompt tone, and the visual warning can be a slightly flashing icon on the dashboard; when the fatigue level is heavy, the sound warning becomes a strong alarm sound, the visual warning becomes a continuously flashing red warning light, and the intensity of the vibration warning is also increased accordingly.

[0080] Time Management:

[0081] After the warning is triggered, the time management submodule immediately starts the timer. During the preset time, it continuously monitors and ensures that the operation restrictions of the steering wheel, brake, and accelerator remain in effect. At the same time, the sound, visual, and vibration warning methods also continue to work, constantly reminding the driver. When the preset time is over, a command to lift the operation restrictions is sent to the intelligent driving system to restore the vehicle to normal operation, and gradually weaken or turn off the sound, visual, and vibration warnings. For example, the preset time is 2 seconds. During this period of time, if the driver takes a rest or other measures to relieve fatigue, the system can lift the operation restrictions and warnings in advance according to changes in physiological signals; if the driver's fatigue state does not improve, the system will continue to maintain the operation restrictions and warnings until the driver regains consciousness.

[0082] 4. Data storage and transmission

[0083] The data storage and transmission module stores the collected physiological signals and analysis results in the local memory for subsequent analysis and query. At the same time, the physiological signals and analysis results are transmitted to the cloud server through wireless communication technology to achieve remote monitoring and management. For example, when the vehicle returns to the garage or connects to the wireless network, the system automatically uploads the physiological signals and analysis results stored locally to the cloud server for professionals to conduct further data analysis and model optimization.

[0084] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology is characterized by: The system includes a physiological signal sensor module, a signal processing and analysis module, an early warning module, and a data storage and transmission module. Specifically: The physiological signal sensor module includes: an electroencephalogram sensor for collecting the driver's electroencephalogram signal; an electrocardiogram sensor for monitoring the driver's heart rate changes; an eye movement sensor for detecting the driver's eye movement; and a skin electrode sensor for measuring the driver's skin electrode response. The signal processing and analysis module includes a signal preprocessing unit, a feature extraction unit, a fusion analysis unit and a personalized physiological model construction unit; wherein the signal preprocessing unit filters, amplifies and denoises the physiological signals collected by the physiological signal sensor module; the feature extraction unit extracts key feature parameters that can reflect the driver's fatigue state from the preprocessed physiological signals; the fusion analysis unit uses a weighted fusion algorithm to comprehensively analyze the key feature parameters of different physiological signals and verify each other; the personalized physiological model construction unit uses the physiological signal data of the driver in a specific time period collected in the early stage and the driver's personal information to construct a personalized physiological model through a machine learning algorithm; and continuously optimizes and updates the model as new data is continuously input during use, specifically: The method for constructing the personalized physiological model selects a support vector machine as a machine learning algorithm, adopts a radial basis function kernel function, uses training data to train the SVM model, solves the model parameters through a sequence minimum optimization algorithm, inputs the physiological signal data into the trained personalized physiological model, and judges the driver's fatigue state through a decision function, which is expressed as follows: in is the decision function value, used to determine the input feature vector The corresponding fatigue state, The input feature vector is composed of key feature parameters of multiple physiological signals. is the weight vector, is the bias term; The optimization method of the personalized physiological model includes adopting cross-validation and introducing regularization. The cross-validation randomly divides the data set into K subsets of equal size; each time one of the subsets is selected as the validation set, and the remaining K-1 subsets are used as the training set; the model training and validation are repeated K times; K validation results are obtained, and the average value is taken as the final model performance evaluation indicator; the regularization determines the loss function with a regularization term: in is the total loss function, which is used to measure the performance of the model. is the original loss function, which is the difference between the model prediction value and the true value. is a regular term used to prevent the model from overfitting. is the regularization parameter, which is used to control the weight of the regularization term in the total loss function; The updating method of the personalized physiological model adopts online learning, uses the stochastic gradient descent algorithm to update the model, calculates the gradient of the loss function to the model parameters, and the gradient represents the change direction and rate of the loss function at the current parameter. According to the formula Update the model parameters, where is the updated parameter, is the current parameter, is the learning rate, which controls the step size of parameter updates, The loss function is at the current parameter The gradient at , indicates the direction and rate of change of the loss function at that point; The warning module includes a communication and coordination submodule, a warning trigger and control submodule and a time management submodule, wherein the communication and coordination submodule establishes communication with the intelligent driving system of the driven vehicle, can send instructions to the intelligent driving system and receive vehicle status information fed back by the intelligent driving system; and coordinates the triggering timing of sound, visual and vibration warnings at the same time; the warning trigger and control submodule determines the driver's fatigue state according to the physiological model constructed by the signal processing and analysis module, and not only controls the triggering of sending control instructions to the intelligent driving system, starting the vehicle speed adjustment and operation restriction functions, but also controls the triggering of sound warnings, visual warnings and vibration warnings at the same time; the time management submodule controls the time process after the warning is issued, starts timing after the warning is issued, continuously executes the operation restriction within the preset time and maintains continuous warnings, and sends a release instruction to the intelligent driving system after the preset time ends to restore the normal operation state of the vehicle, while gradually weakening or turning off the sound, visual and vibration warnings; The data storage and transmission module includes two parts: data storage and data transmission. The data storage stores the collected physiological signals and analysis results in the local memory; the data transmission transmits the physiological signals and analysis results to the cloud server through wireless communication technology.

2. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 1 is characterized by: The EEG sensor is an elastic headband with built-in sensor electrodes, which is worn on the forehead and detects changes in electrical signals generated by brain neuron activity through contact between the electrodes and the scalp; the ECG sensor and skin conductivity sensor are integrated on an elastic wristband, worn on the wrist, and record changes in electrical signals generated by each heart beat through electrodes, thereby monitoring the heart rate and changes in heart rate, and measuring changes in skin resistance; the eye movement sensor is installed near the dashboard directly in front of the driver or at the rearview mirror inside the car, and uses optical principles or infrared technology to detect the driver's eye position, movement trajectory and blinking frequency.

3. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 1 is characterized by: The specific processing steps of the signal preprocessing unit include: ①Select Daubechies wavelet basis function; ②Determine the number of decomposition layers according to the frequency range of the signal and the frequency components that need to be filtered; ③ Perform multi-scale decomposition on the signal to obtain wavelet coefficients and extract physiological signal components in different frequency ranges; ④ For filtering operation, select the corresponding scale of the retained frequency components according to the filtering requirements, and reconstruct the filtered signal through inverse transformation; ⑤ For denoising operation, soft threshold or hard threshold method is used to perform threshold processing on wavelet coefficients, and then the denoised signal is reconstructed through inverse transformation; ⑥ For the amplification operation, determine the frequency scale to be amplified, amplify the corresponding scale coefficient by multiplying it by the amplification factor, and finally reconstruct the amplified signal by inverse transformation.

4. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 3 is characterized by: The key feature parameters extracted by the feature extraction unit are divided into: ① For EEG sensor signals, extract the power spectrum density and focus on analyzing the power changes of α, β, θ, and δ waves in different frequency bands; ② For the ECG sensor signal, extract the heart rate variability index, including the standard deviation of the intervals between adjacent heartbeats, and extract the average heart rate and heart rate change trend; ③ For the eye movement sensor signal, extract the blink frequency, extract the fixation time, including the average fixation time and the stability of the fixation time, and also extract the scan amplitude and speed; ④ For the skin electrical sensor signal, extract the skin electrical response amplitude and extract the skin electrical response change rate.

5. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 4 is characterized by: The weighted fusion algorithm adopted by the fusion analysis unit assigns different weights to each physiological signal according to its importance, and then fuses the weighted key feature parameters. First, the weight of each physiological signal is determined, and then the key feature parameters of each physiological signal are multiplied by the corresponding weight to obtain the weighted key feature parameters; finally, the weighted key feature parameters are summed to obtain the fused feature value, and the specific expression is: in is the fused eigenvalue, is the number of physiological signals, For the The weight of the physiological signal, For the characteristic parameters of a physiological signal.

6. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 1 is characterized by: The communication and coordination submodule selects Bluetooth, Wi-Fi or vehicle communication protocol to establish a connection channel, continuously monitors the output results of the signal processing and analysis module, and prepares to send instructions to the intelligent driving system once it determines that the driver is in a fatigue state; at the same time, it receives vehicle status information feedback from the intelligent driving system, and coordinates the triggering timing of sound, visual and vibration warnings based on fatigue status judgment and vehicle status information.

7. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 6 is characterized by: The warning trigger and control submodule continuously receives the fatigue state judgment result from the signal processing and analysis module. When it is judged that the driver is in a fatigue state, it immediately sends a control instruction to the intelligent driving system to start the vehicle speed adjustment and operation restriction functions. At the same time, it triggers the sound warning, visual warning and vibration warning, and adjusts the intensity and combination of various warning methods according to the degree of fatigue; among which the sound warning is the in-car speaker in the cab, the visual warning is the warning icon or flashing light displayed on the instrument panel, and the vibration warning is the vibration of the seat or steering wheel.

8. The intelligent early warning system for fatigue driving based on physiological signal monitoring technology according to claim 7 is characterized by: After the warning is triggered, the time management submodule immediately starts the timer to start timing, and continuously monitors and ensures that the operating restrictions of the steering wheel, brake, and accelerator remain in effect within the preset time; at the same time, the sound, visual, and vibration warning methods also continue to work to constantly remind the driver; when the preset time is over, an instruction to lift the operating restrictions is sent to the intelligent driving system to restore the vehicle to normal operating status, and the sound, visual, and vibration warnings are gradually weakened or turned off.

Images