Fatigue driving monitoring system based on various feature information

Abstract

The invention relates to the field of information monitoring, and discloses a fatigue driving monitoring system based on various feature information, which is used for solving the problem that a driver can continue to receive an order on another platform when the driver reaches a fatigue driving duration threshold on one platform. Comprising a sensor module, a data acquisition module, a processing module, a data model module, a decision module and an early warning module, and is used for monitoring a physiological signal of a driver and a vehicle driving state in real time, preprocessing the acquired data, learning and modeling the preprocessed data by using deep learning and machine learning algorithms, and performing early warning. Meanwhile, data are processed and analyzed, features related to fatigue driving are extracted, daily driving habits and behavior modes of a driver are recorded, a driving behavior model is established, the fatigue state of the driver is comprehensively evaluated, early warning is given out according to the fatigue state of the driver, management of fatigue driving of the driver is effectively enhanced, and the driving safety is improved. And the safety of drivers and passengers is improved.

Description

Technical Field

[0001] The present invention relates to the field of information monitoring, and more specifically to a fatigue driving monitoring system based on multiple characteristic information. Background Art

[0002] Fatigue driving is one of the important causes of traffic accidents. Fatigue driving refers to driving behavior in which the driver becomes mentally and physically fatigued due to long-term driving, lack of adequate sleep, poor physical condition, etc., which affects his attention, reaction ability and judgment on road traffic. Fatigue driving is a dangerous behavior that increases the risk of traffic accidents and may lead to casualties in severe cases.

[0003] Existing fatigue driving detection methods are mainly based on a single monitoring method of vehicle behavior characteristics, such as driving distance, driving time, lane departure, etc. However, this detection method will result in the lack of an information sharing mechanism between the various platforms. When online ride-hailing drivers register to accept orders on multiple platforms, after reaching the fatigue driving time threshold on one platform, they can continue to accept orders on another platform.

[0004] In view of the above problems, the present invention proposes a solution. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a fatigue driving monitoring system based on multiple characteristic information to solve the problems existing in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A fatigue driving monitoring system based on multiple characteristic information comprises the following steps:

[0008] It includes sensor module, data acquisition module, processing module, data model module, decision module and early warning module. The modules are connected by signals. The steps of data processing between modules are as follows:

[0009] Real-time monitoring of the driver's physiological signals and vehicle driving status through sensor modules;

[0010] The data collected by the sensor module is collected through the data collection module, and the collected data is pre-processed to ensure the quality and consistency of the data;

[0011] Through the processing module, the convolutional neural network is used to extract features from the driver's facial expressions and eye images, and the support vector machine algorithm is used to classify and identify the extracted features, and each classification category is assigned a value. By identifying the current driver's face and eye categories, the facial fatigue coefficient is obtained, and the driver's conversation activity is judged using speech recognition technology;

[0012] The data model module records the driver's daily driving habits and behavior patterns, establishes a driving behavior model, and compares it with the model to determine whether the driver has a tendency to drive fatigued;

[0013] The driver's fatigue state is comprehensively evaluated by the decision module in combination with the output of the processing module and the record of the data model module, and the comprehensive evaluation result is transmitted to the early warning module;

[0014] The early warning module issues early warnings or takes intervention measures based on the comprehensive evaluation results of the decision-making module.

[0015] Preferably, the step of preprocessing the collected data is:

[0016] Select a suitable filter according to the characteristics of the data, and use the filter to filter the data;

[0017] The wavelet transform noise reduction technology is used to reduce the noise of the collected data;

[0018] The data were normalized using the z-score standardization method to have the same scale and range.

[0019] Preferably, the step of normalizing the data using the z-score standardization method is:

[0020] The collected data is taken as a data set and recorded as a sample, and the mean and standard deviation of all samples are calculated. The calculation formulas are Where JZ represents the mean of the sample, BZ represents the standard deviation of the sample, and x i Represents each sample value, n is the number of samples;

[0021] Subtract the mean of the feature from all its sample values ​​and divide it by the standard deviation of the feature. The calculation formula is: Where Z i represents the standardized sample value, JZ represents the mean of the sample, BZ represents the standard deviation of the sample, x i Represented as each sample value;

[0022] For each data in the data set, the standardized value of its Z-score is calculated according to the above steps.

[0023] Preferably, the step of extracting features from the driver's facial expression and eye image using a convolutional neural network is:

[0024] Pre-process the collected driver facial expressions and eye images to ensure the consistency and quality of input data;

[0025] Build a CNN model to extract features from facial expressions and eye images;

[0026] The preprocessed facial expression and eye images are used as the input of the CNN model. The image data is passed through the network through the forward propagation process, and features are extracted in the convolution layer. The pooling layer reduces the dimension of the output of the convolution layer, reduces the number of parameters, and improves the computational efficiency of the model.

[0027] Before the fully connected layer, the feature map output by the convolutional layer is flattened into a vector as the input of the fully connected layer. The fully connected layer linearly combines the features through the weight matrix and the bias vector, and applies the activation function to obtain the final feature representation;

[0028] The CNN model is trained using labeled training data, and the model parameters are optimized using the cross entropy loss function.

[0029] In the trained CNN model, the corresponding feature representation is obtained by inputting facial expressions and eye images into the model.

[0030] Preferably, the step of using speech recognition technology to determine the driver's conversation activity is:

[0031] The driver's voice data is collected through sensors and pre-processed, including noise reduction, removal of silent segments, and voice segmentation;

[0032] Use speech recognition technology to convert the driver's speech into text, and use natural language processing technology to perform sentiment analysis on the converted text to obtain the driver's emotional state;

[0033] Perform speech feature analysis on the converted text to obtain information such as speech speed and intonation;

[0034] According to the importance, the driver's emotional state, speaking speed and tone are weighted. Based on the obtained driver's emotional state, speaking speed and tone, the conversation activity is comprehensively calculated. The calculation formula is: Among them, CA represents the conversation activity, MO represents the driver's emotional state, SE represents the driver's speaking speed, and IT represents the driver's tone of voice.

[0035] Preferably, the step of judging whether the driver has a tendency to drive fatigued by comparing with the model is to collect the behavioral characteristics of the current driver and generate a current driving behavior model, and calculate the current driving behavior model with the standard driving behavior model to obtain a behavioral fatigue coefficient, the calculation formula of which is BF=f(CT, SD), wherein BF represents the behavioral fatigue coefficient, CT represents the current driving behavior model, and SD represents the standard driving behavior model.

[0036] Preferably, the step of comprehensively evaluating the driver's fatigue state by combining the output of the processing module and the record of the data model module is to calculate the driver's fatigue index by weighted summation of the conversation activity, the facial fatigue coefficient and the behavioral fatigue coefficient, and the calculation formula is: DF represents the driver fatigue index, FG represents the facial fatigue coefficient, CA represents the conversation activity, BF represents the behavioral fatigue coefficient, and a1, a2, and a3 represent the weight coefficients of the facial fatigue coefficient, the conversation activity, and the behavioral fatigue coefficient.

[0037] Preferably, the step of issuing a warning or taking intervention measures based on the comprehensive evaluation results of the decision module is to compare the driver fatigue index obtained by comprehensive calculation with a preset threshold value. If the driver fatigue index is less than the preset threshold value, it is determined that the current driver is not in a fatigue state and there is no need to issue a warning reminder to the driver. If the driver fatigue index is greater than the preset threshold value, it is determined that the current driver is in a fatigue state, and a fatigue reminder is given through voice broadcast to remind the driver that he needs to rest, and the platform is notified to let the platform restrict the driver from accepting orders, freeze accounts, and other measures to prevent the driver from accepting orders on this platform or other platforms.

[0038] Technical effects and advantages of the present invention:

[0039] It monitors the driver's physiological signals and vehicle driving status in real time, pre-processes the collected data, uses deep learning and machine learning algorithms to learn and model the pre-processed data, processes and analyzes the data at the same time, extracts features related to fatigue driving, records the driver's daily driving habits and behavior patterns, establishes a driving behavior model, comprehensively evaluates the driver's fatigue status, and issues early warnings based on the driver's fatigue status, effectively strengthening the management of driver fatigue driving and improving the safety of drivers and passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall flow chart of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. In addition, the forms of the various structures recorded in the following embodiments are merely illustrative. The fatigue driving monitoring system based on multiple feature information involved in the present invention is not limited to the various structures recorded in the following embodiments. All other implementations obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0042] The present invention provides a fatigue driving monitoring system based on multiple characteristic information, including a sensor module, a data acquisition module, a processing module, a data model module, a decision module and an early warning module. The modules are connected by signals, and the data processing steps between the modules are as follows:

[0043] The sensor module is used to monitor the driver's physiological signals and vehicle driving status in real time. The physiological signals include facial expressions, pupil size, blinking frequency, single eye closure duration, yawning, head posture, conversation activity and other parameters. The vehicle driving status includes driving time, vehicle speed, acceleration, steering wheel angle and braking distance information. The use of sensor modules to collect the driver's physiological signals and vehicle driving status data in real time can ensure that the data collection process is stable and reliable, and can transmit data to the monitoring system in a timely manner.

[0044] In this embodiment, it should be specifically explained that the steps of monitoring the driver's physiological signals and the vehicle driving status are:

[0045] Select appropriate sensors to monitor the driver's physiological signals and vehicle driving status. Common physiological signal sensors include heart rate sensors, skin resistance sensors, eye tracking sensors, etc. Vehicle driving status sensors include speed sensors, steering sensors, acceleration sensors, etc., and ensure that the selected sensors can accurately and stably obtain the required data;

[0046] Install the selected sensor modules in the cockpit to accurately monitor the driver's physiological signals and vehicle driving status. For example, a heart rate sensor can be installed above the driver's seat, and an eye tracking sensor can be installed at the front of the vehicle or on the driver's face;

[0047] Calibrate the installed sensors to ensure that they can accurately acquire data and match the actual situation. Calibration usually needs to be performed in actual driving environments to take into account different driving conditions and driver characteristics.

[0048] The data collected by the sensor module is collected through the data acquisition module, and the collected data is pre-processed, such as filtering, denoising and normalizing the signal, to ensure the quality and consistency of the data.

[0049] In this embodiment, it should be specifically explained that the steps of pre-processing the collected data are:

[0050] For physiological signal data, such as heart rate, skin resistance, etc., there may be noise interference, so filters are used to filter the data. Common filtering methods include low-pass filtering, high-pass filtering and band-pass filtering. Choose the appropriate filter according to the characteristics of the data;

[0051] The low-pass filter, high-pass filter and band-pass filter are three types of filters commonly used in signal processing. They are used to selectively pass or suppress frequency components of signals within different frequency ranges. The low-pass filter filters out the high-frequency part and retains the low-frequency components in the signal, allowing signal components below a certain cut-off frequency to pass through, but blocking signal components above this frequency, and is used to remove high-frequency noise, smooth signals, and reduce signal fluctuations; the high-pass filter filters out the low-frequency part and retains the high-frequency components in the signal, and is often used to remove low-frequency noise, detect signal changes, and emphasize the high-frequency part of the signal; the band-pass filter selects a signal component within a specific frequency band, and the band-pass filter limits the frequency range to a specific bandwidth, allowing signal components within the frequency band to pass through, but blocking signal components in other frequency ranges, and is used to extract signals within a specific frequency range, such as speech signal processing, electrocardiogram signal analysis, etc.

[0052] The collected data is processed by wavelet transform noise reduction technology. During the data collection process, it may be interfered by the external environment, such as electromagnetic interference, mechanical vibration, etc.

[0053] Wavelet transform denoising technology is a commonly used signal processing method used to reduce the noise of signals. It uses the characteristics of wavelet transform to decompose and reconstruct signals in the time domain and frequency domain to achieve the purpose of removing noise.

[0054] The steps of using wavelet transform noise reduction technology to reduce noise of collected data are as follows:

[0055] The signal to be processed is decomposed into wavelet coefficients of different frequencies through wavelet transform. Wavelet transform has the characteristics of time-frequency locality and can better capture the time domain and frequency domain characteristics of the signal;

[0056] Threshold processing is performed on the wavelet coefficients to set the smaller wavelet coefficients to zero or scale them. The smaller wavelet coefficients are usually considered as noise. The purpose of threshold processing is to remove or weaken the noise coefficients while retaining important information in the signal;

[0057] The wavelet coefficients after threshold processing are reconstructed through inverse wavelet transform to obtain the denoised signal. The inverse wavelet transform resynthesizes the processed wavelet coefficients into a signal in the time domain. The inverse wavelet transform is the inverse process of the wavelet transform, which is used to resynthesize the wavelet coefficients into the original signal. In wavelet analysis, the signal will be decomposed into wavelet coefficients of different scales and frequencies after wavelet transform, and the inverse wavelet transform is the process of resynthesizing these wavelet coefficients into the original signal.

[0058] Use the z-score standardization method to normalize the data to have the same scale and range. This ensures that the data collected by different sensors are comparable and can improve the convergence speed and accuracy of subsequent models;

[0059] The steps of normalizing the data using the z-score standardization method are as follows:

[0060] The collected data is taken as a data set and recorded as a sample, and the mean and standard deviation of all samples are calculated. The calculation formulas are Where JZ represents the mean of the sample, BZ represents the standard deviation of the sample, and x i Represents each sample value, n is the number of samples;

[0061] Subtract the mean of the feature from all its sample values ​​and divide it by the standard deviation of the feature. The calculation formula is: Where Z i represents the standardized sample value, JZ represents the mean of the sample, BZ represents the standard deviation of the sample, x i Represented as each sample value;

[0062] For each data in the data set, the standardized value of its Z-score is calculated according to the above steps.

[0063] The processing module uses deep learning and machine learning algorithms to learn and model the pre-processed data, and processes and analyzes the data to extract features related to fatigue driving.

[0064] The method uses deep learning and machine learning algorithms to learn and model the preprocessed data, and processes and analyzes the data to extract features related to fatigue driving, using a convolutional neural network to extract features of the driver's facial expressions and eye images, using a support vector machine (SVM) algorithm to classify and identify the extracted features, and assigning values ​​to each classification category. By identifying the current driver's face and eye categories, the facial fatigue coefficient is obtained, and the driver's conversation activity is judged using speech recognition (ASR) technology.

[0065] In this embodiment, it should be specifically explained that the steps of extracting features of the driver's facial expression and eye image using a convolutional neural network are as follows:

[0066] Preprocess the collected driver's facial expression and eye images, such as cropping, resizing, graying, etc., to ensure the consistency and quality of the input data;

[0067] Build a CNN model to extract features from facial expressions and eye images. CNN usually includes convolutional layers, pooling layers, and fully connected layers. Convolutional layers are used to extract local features, pooling layers are used to reduce data dimensions, and fully connected layers are used for classification or regression tasks.

[0068] The preprocessed facial expression and eye images are used as the input of the CNN model. The image data is passed through the network through the forward propagation process, and features are extracted in the convolution layer. The pooling layer reduces the dimension of the output of the convolution layer, reduces the number of parameters, and improves the computational efficiency of the model.

[0069] Before the fully connected layer, the feature map output by the convolutional layer is flattened into a vector as the input of the fully connected layer. The fully connected layer linearly combines the features through the weight matrix and the bias vector, and applies the activation function to obtain the final feature representation;

[0070] The CNN model is trained using labeled training data, and the model parameters are optimized using the cross entropy loss function, which is an indicator used to measure the difference between two probability distributions and is commonly used in classification tasks. In deep learning, the cross entropy loss function is usually used to measure the difference between the probability distribution output by the model and the true label, so as to be used for model optimization training. Its function is defined as Where N is the number of samples, y i is the true label of the i-th sample, The probability of the positive class of the i-th sample predicted by the model;

[0071] In the trained CNN model, by inputting facial expressions and eye images into the model, the corresponding feature representations can be obtained, which can be used for subsequent tasks such as emotion recognition, fatigue detection, etc.

[0072] After using convolutional neural networks to extract features, we obtain a set of feature vectors that describe the abstract representation of each sample at a high level, so we need to use the support vector machine algorithm to classify and identify these feature vectors.

[0073] In this embodiment, it should be specifically explained that the steps of classifying and identifying the extracted features using the support vector machine (SVM) algorithm are as follows:

[0074] Prepare training data sets and test data sets. Each sample has a feature vector and a corresponding label, which indicates which category the sample belongs to;

[0075] Normalize the features so that they have similar scales. This can help the algorithm converge better and improve the performance of the model;

[0076] Using the training data set, the support vector machine algorithm will try to find an optimal hyperplane to separate samples of different categories. Its goal is to find a hyperplane that maximizes the margin and establishes a clear boundary between different categories.

[0077] After training the model, we can use it to classify and predict the samples in the test set. For each test sample, SVM will classify it into a category based on its feature vector.

[0078] In this embodiment, it should be specifically explained that the step of using speech recognition technology to determine the driver's conversation activity is:

[0079] The driver's voice data is collected through sensors and pre-processed, including noise reduction, removal of silent segments, and voice segmentation;

[0080] Use speech recognition technology to convert the driver's speech into text, and use natural language processing technology to perform sentiment analysis on the converted text to obtain the driver's emotional state;

[0081] Perform speech feature analysis on the converted text to obtain information such as speaking rate and intonation. The speaking rate can be estimated by counting the number of syllables or the average syllable length in the text. The intonation can be obtained by analyzing the stress and pitch changes in the text;

[0082] According to the importance, the driver's emotional state, speaking speed and tone are weighted. Based on the obtained driver's emotional state, speaking speed and tone, the conversation activity is comprehensively calculated. The calculation formula is: Among them, CA represents the conversation activity, MO represents the driver's emotional state, SE represents the driver's speaking speed, and IT represents the driver's tone of voice.

[0083] Through the data model module, the driver's daily driving habits and behavior patterns are recorded, a driving behavior model is established, and by comparing with the model, it is determined whether the driver has a tendency to fatigue driving.

[0084] In this embodiment, it should be specifically explained that the steps of establishing a driving behavior model by recording the driver's daily driving habits and behavior patterns are as follows:

[0085] Collect the driver's daily driving data. This data includes vehicle sensor data, such as speed, acceleration, dashcam data, and GPS data;

[0086] Preprocessing the collected data, including data cleaning, removing outliers, filling missing values, etc. Ensuring the quality and integrity of the data is the key to establishing a reliable driving behavior model;

[0087] Extract features from the preprocessed data, which can reflect the driver's driving habits and behavior patterns. Features may include but are not limited to: driving speed, acceleration, turning radius, driving time, stop time, driving route, frequent lane changes, frequent braking, etc.;

[0088] A driving behavior model is established using a support vector machine model, and the extracted features and corresponding labels are used for training to obtain a standard driving behavior model.

[0089] The method of judging whether the driver has a tendency of fatigue driving by comparing with the model is to collect the behavioral characteristics of the current driver and generate a current driving behavior model, and calculate the current driving behavior model and the standard driving behavior model to obtain a behavioral fatigue coefficient, and the calculation formula is BF=f(CT, SD), wherein BF represents the behavioral fatigue coefficient, CT represents the current driving behavior model, and SD represents the standard driving behavior model.

[0090] The driver's fatigue state is comprehensively judged by the decision module in combination with the output of the processing module and the data model module record. For example, if the processing module determines that the driver is in a fatigue state, the decision module can pass the judgment result to the early warning module.

[0091] In this embodiment, it should be specifically explained that the step of comprehensively judging the driver's fatigue state by combining the output of the processing module and the record of the data model module is to calculate the driver's fatigue index by weighted summation of the conversation activity, facial fatigue coefficient and behavioral fatigue coefficient output by the processing module, and the calculation formula is: DF stands for driver fatigue index, FG stands for facial fatigue coefficient, which is considered an indicator of driver fatigue status, and fatigue driving often leads to facial fatigue. When a driver is fatigued, some facial features may appear, such as red eyes, heavy eyelids, and reduced blinking frequency, which can be regarded as manifestations of facial fatigue. CA stands for conversation activity. When drivers are fatigued, their attention may decrease, and more attention is used to maintain driving tasks rather than to have conversations. This may lead to a decrease in conversation activity. Fatigue can affect the driver's reaction speed and mental agility, making them more sluggish and inactive in conversations. BF stands for behavioral fatigue coefficient. Fatigue driving often leads to changes in the driver's behavioral characteristics, such as slower driving speed, frequent lane changes, and sudden braking. These changes in behavioral characteristics can be considered as a manifestation of the behavioral fatigue coefficient. When in a fatigued state, the driver's attention and reaction ability will decrease, and they are prone to dangerous behaviors. Therefore, the higher the fatigue level, the more unsafe behaviors the driver may exhibit, thereby increasing the behavioral fatigue coefficient. a1, a2, and a3 are expressed as weight coefficients of facial fatigue coefficient, conversation activity, and behavioral fatigue coefficient, and this embodiment does not perform specific calculations on the specific values ​​of a1, a2, and a3.

[0092] The early warning module issues an early warning or takes intervention measures based on the calculation results of the decision-making module.

[0093] In the present embodiment, it should be specifically explained that the step of issuing a warning or taking intervention measures according to the judgment result of the decision-making module is to compare the driver fatigue index obtained by comprehensive calculation with the preset threshold value. If the driver fatigue index is less than the preset threshold value, it is determined that the current driver is not in a fatigue state and there is no need to issue a warning reminder to the driver. If the driver fatigue index is greater than the preset threshold value, it is determined that the current driver is in a fatigue state, and a fatigue reminder is issued through voice broadcast to remind the driver that he needs to rest, and the platform is notified to let the platform restrict the driver from accepting orders, freeze accounts and other measures to prevent the driver from accepting orders on this platform or other platforms.

[0094] The early warning system can detect signs of fatigue driving in time and reduce the risk of traffic accidents caused by fatigue driving, thereby ensuring the safety of passengers. Fatigue driving not only poses a threat to the safety of passengers, but also poses a danger to the safety of the driver himself. Forcing drivers to be unable to accept orders can prevent drivers from getting into traffic accidents due to fatigue driving and protect their own safety. Online ride-hailing platforms issue early warnings for fatigue driving and take corresponding measures, which helps the platforms operate in compliance with regulations, meet the requirements of traffic safety laws and regulations, and enhance the social responsibility and public trust of the platforms. Fatigue driving may cause drivers to decline in driving skills, making them more likely to have traffic accidents or traffic violations, which in turn leads to traffic congestion. Through the early warning system, fatigue driving can be detected in time and drivers can be prevented from continuing to accept orders, which helps reduce traffic congestion.

[0095] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0096] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A fatigue driving monitoring system based on multiple feature information, characterized in that: It includes sensor module, data acquisition module, processing module, data model module, decision module and early warning module. The modules are connected by signals. The steps of data processing between modules are as follows: Real-time monitoring of the driver's physiological signals and vehicle driving status through sensor modules; The data collected by the sensor module is collected through the data collection module, and the collected data is pre-processed to ensure the quality and consistency of the data; Through the processing module, the convolutional neural network is used to extract features from the driver's facial expressions and eye images, and the support vector machine algorithm is used to classify and identify the extracted features, and each classification category is assigned a value. By identifying the current driver's face and eye categories, the facial fatigue coefficient is obtained, and the driver's conversation activity is judged using speech recognition technology; The data model module records the driver's daily driving habits and behavior patterns, establishes a driving behavior model, and compares it with the model to determine whether the driver has a tendency to drive fatigued; The driver's fatigue state is comprehensively evaluated by the decision module in combination with the output of the processing module and the record of the data model module, and the comprehensive evaluation result is transmitted to the early warning module; The early warning module issues early warnings or takes intervention measures based on the comprehensive evaluation results of the decision-making module.

2. A fatigue driving monitoring system based on multiple characteristic information according to claim 1, characterized in that: The steps of preprocessing the collected data are as follows: Select a suitable filter according to the characteristics of the data, and use the filter to filter the data; The wavelet transform noise reduction technology is used to reduce the noise of the collected data; The data were normalized using the z-score standardization method to have the same scale and range.

3. A fatigue driving monitoring system based on multiple characteristic information according to claim 2, characterized in that: The steps of normalizing the data using the z-score standardization method are as follows: The collected data is taken as a data set and recorded as a sample, and the mean and standard deviation of all samples are calculated. The calculation formulas are Where JZ represents the mean of the sample, BZ represents the standard deviation of the sample, and x i Represents each sample value, n is the number of samples; Subtract the mean of the feature from all its sample values ​​and divide it by the standard deviation of the feature. The calculation formula is: Where Z i represents the standardized sample value, JZ represents the mean of the sample, BZ represents the standard deviation of the sample, x i Represented as each sample value; For each data in the data set, the standardized value of its Z-score is calculated according to the above steps.

4. The fatigue driving monitoring system based on multiple characteristic information according to claim 1 is characterized in that: The steps of extracting features from the driver's facial expression and eye image using a convolutional neural network are as follows: Pre-process the collected driver facial expressions and eye images to ensure the consistency and quality of input data; Build a CNN model to extract features from facial expressions and eye images; The preprocessed facial expression and eye images are used as the input of the CNN model. The image data is passed through the network through the forward propagation process, and features are extracted in the convolution layer. The pooling layer reduces the dimension of the output of the convolution layer, reduces the number of parameters, and improves the computational efficiency of the model. Before the fully connected layer, the feature map output by the convolutional layer is flattened into a vector as the input of the fully connected layer. The fully connected layer linearly combines the features through the weight matrix and the bias vector, and applies the activation function to obtain the final feature representation; The CNN model is trained using labeled training data, and the model parameters are optimized using the cross entropy loss function. In the trained CNN model, the corresponding feature representation is obtained by inputting facial expressions and eye images into the model.

5. The fatigue driving monitoring system based on multiple characteristic information according to claim 1 is characterized in that: The steps of using speech recognition technology to determine the driver's conversation activity are: The driver's voice data is collected through sensors, and the collected voice data is pre-processed, including noise reduction, removal of silent segments, and voice segmentation; Use speech recognition technology to convert the driver's speech into text, and use natural language processing technology to perform sentiment analysis on the converted text to obtain the driver's emotional state; Perform speech feature analysis on the converted text to obtain information such as speech speed and intonation; According to the importance, the driver's emotional state, speaking speed and tone are weighted. Based on the obtained driver's emotional state, speaking speed and tone, the conversation activity is comprehensively calculated. The calculation formula is: Among them, CA represents the conversation activity, MO represents the driver's emotional state, SE represents the driver's speaking speed, and IT represents the driver's tone of voice.

6. The fatigue driving monitoring system based on multiple characteristic information according to claim 1 is characterized in that: The step of judging whether the driver has a tendency to drive fatigued by comparing with the model is to collect the behavioral characteristics of the current driver and generate a current driving behavior model, and calculate the current driving behavior model and the standard driving behavior model to obtain a behavioral fatigue coefficient, the calculation formula of which is BF=f(CT, SD), wherein BF represents the behavioral fatigue coefficient, CT represents the current driving behavior model, and SD represents the standard driving behavior model.

7. The fatigue driving monitoring system based on multiple characteristic information according to claim 1 is characterized in that: The step of comprehensively evaluating the driver's fatigue state by combining the output of the processing module and the record of the data model module is to calculate the driver's fatigue index by weighted summation of the conversation activity, the facial fatigue coefficient and the behavioral fatigue coefficient, and the calculation formula is: DF represents the driver fatigue index, FG represents the facial fatigue coefficient, CA represents the conversation activity, BF represents the behavioral fatigue coefficient, and a1, a2, and a3 represent the weight coefficients of the facial fatigue coefficient, the conversation activity, and the behavioral fatigue coefficient.

8. The fatigue driving monitoring system based on multiple characteristic information according to claim 1 is characterized in that: The step of issuing a warning or taking intervention measures based on the comprehensive evaluation results of the decision-making module is to compare the driver fatigue index obtained by comprehensive calculation with the preset threshold value. If the driver fatigue index is less than the preset threshold value, it is determined that the current driver is not in a fatigue state and there is no need to issue a warning reminder to the driver. If the driver fatigue index is greater than the preset threshold value, it is determined that the current driver is in a fatigue state, and a fatigue reminder is given through voice broadcast to remind the driver that he needs to rest, and the platform is notified to let the platform restrict the driver from accepting orders, freeze the account and other measures to prevent the driver from accepting orders on this platform or other platforms.

Images