A method and system for safe driving assessment based on in-vehicle microwave signals and fatigue signals

Abstract

This invention belongs to the field of active driving safety technology and provides a method and system for judging safe driving based on in-vehicle microwave signals and fatigue signals. The method includes: acquiring the driver's microwave signals, fatigue signals, driving duration, and in-vehicle temperature; and making a safe driving judgment using abnormal driving prediction results and fatigue prediction results. Based on the use of a generalized linear regression analysis model and deep neural network for abnormal driving and fatigue prediction, this invention comprehensively considers microwave signals, fatigue signals, driving duration, and in-vehicle temperature, solving the problem of relying on only one type of alarm information and improving prediction accuracy. Simultaneously, during fatigue prediction, it removes abnormal fatigue signals corresponding to problems with abnormal driving prediction results at the same time, avoiding control confusion caused by simultaneously using abnormal driving prediction results and fatigue driving prediction results, while also reducing the overall data processing volume of the model.

Description

Technical Field

[0001] This invention belongs to the field of active driving safety technology, and in particular relates to a method and system for judging safe driving based on in-vehicle microwave signals and fatigue signals. Background Technology

[0002] Current driver fatigue alerts primarily use an infrared camera facing the driver to monitor details such as the driver's head, eyes, and face in real time. The acquired data is then processed for pattern recognition to determine fatigue or distraction. Driver Monitoring Systems (DMS) can detect driver fatigue, distraction, and unexpected situations that render the driver unable to drive, providing warnings. Current in-vehicle microwave radar is mainly used for rear-seat child detection. After the vehicle is turned off and all four doors and hoods are closed, it activates a liveness detection system. When it detects the characteristics of a child left behind, it notifies the driver and surrounding personnel of the presence of a child through methods such as horn blaring, hazard lights flashing, or SMS notifications. It can also monitor the driver's heart rate, respiratory rate, and fatigue level in real time.

[0003] While current vehicles can monitor occupant fatigue, heart rate, and respiratory rate using infrared cameras and microwave signals, and make corresponding judgments and controls, the alarm information based on fatigue driving and driver illness based on heart rate and respiratory rate signals is relatively simple and rigid. The corresponding alarm, gear shifting, and automatic emergency call control strategies are also relatively simple, failing to effectively classify safety levels and causing problems such as over-control or false alarms. Furthermore, the lack of connection between driver fatigue and illness information and the driving environment increases the probability of accidents when controlling the vehicle. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a safe driving judgment method and system based on in-vehicle microwave signals and fatigue signals. This invention utilizes a generalized linear regression analysis model and deep neural networks for abnormal driving and fatigue prediction. By comprehensively considering microwave signals, fatigue signals, driving duration, and in-vehicle temperature, it solves the problem of relying on a single alarm information, improving prediction accuracy. Furthermore, during fatigue prediction, it eliminates abnormal fatigue signals corresponding to problems with abnormal driving prediction results occurring simultaneously, avoiding control confusion caused by simultaneously using abnormal driving and fatigue driving prediction results, and reducing the overall data processing volume of the model.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solution:

[0006] In a first aspect, the present invention provides a method for determining safe driving based on in-vehicle microwave signals and fatigue signals, including:

[0007] Acquire the driver's microwave signals, fatigue signals, driving duration, and in-vehicle temperature;

[0008] Based on the microwave signal, driving duration, and in-vehicle temperature, as well as a preset abnormal driving linear prediction model, the abnormal driving prediction result is obtained; wherein, the abnormal driving linear prediction model is a generalized linear regression analysis model.

[0009] Based on fatigue signals and a preset fatigue prediction model, the driver fatigue prediction result is determined; wherein, the fatigue prediction model is a deep neural network; during prediction, abnormal fatigue signals corresponding to problems with abnormal driving prediction results within the same instant are eliminated.

[0010] Safe driving judgments are made using abnormal driving prediction results and fatigue prediction results.

[0011] Furthermore, a linear prediction model for abnormal driving:

[0012] y = β0 + β1x1 + β2x2 + ... + β n x n +ε

[0013]

[0014] Where y is the prediction result; β0 is the intercept term; x i The independent variables are i = n, n = 1, 2, 3, and n, which correspond to the values ​​of the independent variables for heart rate, respiratory rate, driving speed, driving duration, and in-vehicle temperature, respectively; β i The independent variable is x i The weighting coefficients; ε is the random error term; X i X represents the measured value of the independent variable. w γ represents the normal reference value for the independent variable; i Adjust the parameters for the independent variable.

[0015] Furthermore, the head contour, mouth contour, and eye contour in the image information are located and cropped; images showing nodding, yawning, closing eyes, and shaking head are marked as abnormal images; in abnormal images, if the abnormal driving prediction result at the corresponding time is yes, the abnormal image at the corresponding time is deleted.

[0016] Furthermore, the error between the deep neural network output and the expected output is calculated, and then the error is backpropagated to the hidden layer to update the weights between neurons in the deep neural network. The weight updates are repeated until the performance of the deep neural network reaches a preset level or the parameters of the deep neural network converge.

[0017] Furthermore, the normal prediction results from the abnormal driving linear prediction model, along with facial expression image information, are used as input to the fatigue prediction model.

[0018] Furthermore, when the abnormal driving prediction result exceeds the preset value, the hazard lights will be activated to issue an alarm, and the vehicle will be stopped and the gear shifted to P during the alarm period; fatigue prediction will not be performed; when the abnormal driving prediction result does not exceed the preset value, fatigue prediction will be performed, and an audible alarm will be issued when a risk of fatigue driving occurs.

[0019] Secondly, the present invention also provides a safe driving judgment system based on in-vehicle microwave signals and fatigue signals, comprising:

[0020] The data acquisition module is configured to acquire the driver's microwave signal, fatigue signal, driving time, and in-vehicle temperature.

[0021] The abnormal driving prediction module is configured to: obtain abnormal driving prediction results based on the microwave signal, driving duration and in-vehicle temperature, and a preset abnormal driving linear prediction model; wherein, the abnormal driving linear prediction model is a generalized linear regression analysis model.

[0022] The fatigue prediction module is configured to: determine the driving fatigue prediction result based on fatigue signals and a preset fatigue prediction model; wherein, the fatigue prediction model is a deep neural network; during prediction, abnormal fatigue signals corresponding to problems with abnormal driving prediction results within the same instant are eliminated;

[0023] The safe driving judgment module is configured to make safe driving judgments based on abnormal driving prediction results and fatigue prediction results.

[0024] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in the first aspect.

[0025] Fourthly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the program to implement the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in the first aspect.

[0026] Fifthly, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in the first aspect.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. Based on the generalized linear regression analysis model and deep neural network for abnormal driving prediction and fatigue prediction, this invention solves the problem of considering only one type of alarm information by comprehensively considering microwave signals, fatigue signals, driving time and in-vehicle temperature, thus improving prediction accuracy. At the same time, in fatigue prediction, abnormal fatigue signals corresponding to problems in abnormal driving prediction results at the same time are removed, avoiding the problem of control confusion caused by making predictions based on both abnormal driving prediction results and fatigue driving prediction results at the same time, and reducing the overall data processing volume of the model.

[0029] 2. In this invention, during the training and prediction of the fatigue prediction model, the normal prediction result from the abnormal driving linear prediction model, along with facial expression image information, is used as the input to the fatigue prediction model. This is equivalent to considering factors such as heart rate, respiratory rate, driving speed, driving duration, and in-vehicle temperature during fatigue prediction, thus improving prediction accuracy. Furthermore, when the abnormal driving prediction result exceeds a preset value, the hazard lights are activated to issue an alarm, and the vehicle is stopped and shifted to P gear during the alarm period; fatigue prediction is not performed. When the abnormal driving prediction result does not exceed the preset value, fatigue prediction is performed. When a risk of fatigue driving occurs, an audible alarm is triggered. By treating the abnormal driving prediction result as a high-level prediction and the fatigue prediction as a relatively low-level prediction, alarms and strategies based on safety level classification are implemented. Attached Figure Description

[0030] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0031] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0032] Figure 2 This is the time axis of Embodiment 1 of the present invention;

[0033] Figure 3 This is a schematic diagram of a deep neural network according to Embodiment 1 of the present invention. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0036] Example 1:

[0037] This embodiment provides a safe driving judgment method based on in-vehicle microwave signals and fatigue signals. By sensing, detecting and tracking the driver's heart rate and fatigue level through in-vehicle microwave radar, and combining the visual function of DMS (fatigue monitoring system), the driver's facial expressions are identified to judge the driver's health status. The judgment results of abnormal breathing rate and heart rate data are output to the instrument warning, then the vehicle is stopped and rescue is called, thereby improving the driver's driving safety.

[0038] Optionally, the method is implemented based on a 66GHz in-vehicle microwave radar (CIR), a fatigue detection system (DMS), an in-vehicle instrument cluster (ICM), a transmission control unit (TCU), an braking system (APB), and a central gateway (CGW). The TCU is responsible for sending gear position signals, the CIR for sending driver heart rate and respiratory rate signals, and the DMS for sending driver fatigue signals. The CIR monitors and sends the driver's health status, the DMS visually detects the driver's facial expressions, and the CIR combines the detected signals with the DMS signals to release the driver's health status signal. The ICM displays the driver's heart rate and respiratory rate and issues an alarm (double flashing lights). The TCU is responsible for sending gear position signals, and the SOS is responsible for sending emergency signals. The CIR uses 60-64GHz, meeting the requirements of most market frequency bands. Two radars are required, installed in the middle of the second-row roof and in the back of the driver's seat, respectively, with a detection range ≥3m. Specifically, the method in this embodiment includes:

[0039] S1. Predicting abnormal driving based on microwave signals:

[0040] Optionally, microwave information collected by microwave radar during normal driving can be used as the normal dataset, and microwave information collected when the driver experiences malfunctions due to illness can be used as the abnormal dataset. The dependent variable is set as whether abnormal driving occurs, with "yes" output when abnormal driving occurs and "no" output otherwise. Optional independent variables include heart rate, respiratory rate, driving speed, driving duration, and in-vehicle temperature. Then, a generalized linear regression analysis is used to obtain the weights of the independent variables, and finally, the probability of abnormal driving during the driving process is predicted based on the obtained weights.

[0041] Specifically, since abnormal driving prediction is a binary outcome ("yes" / "no"), a binomial distribution can be chosen as the family of distributions for the response variable, and logistic regression can be used. In the generalized linear regression model, the link function connects the linear predictor to the expected value of the response variable. The link function uses the logit function to map the probability to all possible real values. Constructing an abnormal driving linear prediction model:

[0042] y = β0 + β1x1 + β2x2 + ... + β n x n +ε

[0043]

[0044] Where y is the prediction result, and when it is greater than the preset value, abnormal driving is predicted; β0 is the intercept term; x i The independent variables are i = n, n = 1, 2, 3, and n, which correspond to the values ​​of the independent variables for heart rate, respiratory rate, driving speed, driving duration, and in-vehicle temperature, respectively; β i The independent variable is x i Weighting coefficients; ∈ represents the random error term; X i X represents the measured value of the independent variable. w γ represents the normal reference value for the independent variable; i The parameters for the independent variables are adjusted by adjusting the ratio of the measured value to the normal reference value so that there is no difference in magnitude between the independent variables after adjustment.

[0045] Optionally, use the statistical R or Python statsmodels library to fit the model.

[0046] S2. Predicting fatigued driving based on fatigue signals:

[0047] S2.1 Optionally, use an infrared camera to collect facial expression images of the driver, including drowsiness, yawning, closed eyes, and head shaking.

[0048] S2.2 Perform feature extraction and filtering on the collected facial expression signals; specifically, locate and crop the head contour, mouth contour, and eye contour in the image information; mark images showing nodding, yawning, closed eyes, and head shaking as abnormal images; among the abnormal images, if the abnormal driving prediction result at the corresponding time is yes, delete the abnormal image at the corresponding time, for example... Figure 2 In the model, if the abnormal driving prediction result y1 exceeds the preset safety value at time t1, it is determined that there is a problem with the driver's physical condition at this time. It is necessary to issue an alarm and take corresponding control measures according to the abnormal driving prediction result. This avoids the problem of control confusion caused by making predictions based on both abnormal driving prediction results and fatigue driving prediction results at the same time, and also reduces the overall data processing volume of the model.

[0049] S2.3, Normalize the facial expression signals:

[0050]

[0051] Where a is the facial expression signal; a' represents the normalized facial expression signal a'∈[0,1]; a maxand a min It represents the maximum and minimum values ​​in facial expression signals; when nodding or shaking the head, the maximum swing scale represents the specific degree of swing, and when yawning or closing the eyes, the frequency of occurrence within a certain time period is used as the specific data to be processed.

[0052] S2.4 The prediction model uses a trained deep neural network (DNN), which includes an input layer, a hidden layer, and an output layer.

[0053] The deep neural network is located as: O∈R m ×R n The model's input is an m-dimensional state vector a = a1, a2, ..., a m The output is an n-dimensional vector O(a). n The computation between neurons is as follows:

[0054]

[0055] in, Z is the activation function of the i-th neuron in layer l; ij This represents the action from neuron i in layer I to neuron j in layer I+1. This is the weight matrix; Let f be the basis function vector of neuron j; f() is the nonlinear softmax function used as the activation function.

[0056] The weight w represents the degree of influence of the input signal on the target neuron. The connections formed between neurons by these weights constitute the network's learning ability, enabling it to adjust these weights through training to fit the relationship between input and output. In this embodiment, the gradient descent algorithm is used to update the model's weight w. Specifically, the error between the model output and the expected output is calculated, and then the error is backpropagated to the hidden layer to update the weight w. This process of updating the weight w is repeated until the model's performance reaches a satisfactory level or the model's parameters have converged.

[0057] The output divisor of the fatigue driving prediction model indicates whether there is a risk of fatigue driving or not.

[0058] During the training and prediction of the fatigue prediction model, the normal prediction result y from the abnormal driving linear prediction model, along with facial expression image information, is used as the input to the fatigue prediction model. This is equivalent to taking into account factors such as heart rate, respiratory rate, driving speed, driving duration, and in-vehicle temperature during fatigue prediction, thereby improving prediction accuracy.

[0059] S3. Utilize abnormal driving prediction results and fatigue prediction results to make safe driving judgments:

[0060] When the abnormal driving prediction result y exceeds the preset value, the hazard lights will be activated to issue an alarm, and the vehicle will be stopped and shifted to P gear during the alarm period. The CAN signal will be triggered to send a message to the E-CALL module to automatically dial the rescue number; fatigue prediction will not be performed.

[0061] When the abnormal driving prediction result y does not exceed the preset value, fatigue prediction is performed; when the risk of fatigue driving occurs, an audible alarm is triggered.

[0062] Example 2:

[0063] This embodiment provides a safe driving judgment system based on in-vehicle microwave signals and fatigue signals, including:

[0064] The data acquisition module is configured to acquire the driver's microwave signal, fatigue signal, driving time, and in-vehicle temperature.

[0065] The abnormal driving prediction module is configured to: obtain abnormal driving prediction results based on the microwave signal, driving duration and in-vehicle temperature, and a preset abnormal driving linear prediction model; wherein, the abnormal driving linear prediction model is a generalized linear regression analysis model.

[0066] The fatigue prediction module is configured to: determine the driving fatigue prediction result based on fatigue signals and a preset fatigue prediction model; wherein, the fatigue prediction model is a deep neural network; during prediction, abnormal fatigue signals corresponding to problems with abnormal driving prediction results within the same instant are eliminated;

[0067] The safe driving judgment module is configured to make safe driving judgments based on abnormal driving prediction results and fatigue prediction results.

[0068] The working method of the system is the same as the safe driving judgment method based on in-vehicle microwave signals and fatigue signals in Embodiment 1, and will not be repeated here.

[0069] Example 3:

[0070] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in Embodiment 1.

[0071] Example 4:

[0072] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in Embodiment 1.

[0073] Example 5:

[0074] This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in Embodiment 1.

[0075] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for judging safe driving based on in-vehicle microwave signals and fatigue signals, characterized in that, include: Acquire the driver's microwave signals, fatigue signals, driving duration, and in-vehicle temperature; Based on the microwave signal, driving duration, and in-vehicle temperature, as well as a preset abnormal driving linear prediction model, the abnormal driving prediction result is obtained; wherein, the abnormal driving linear prediction model is a generalized linear regression analysis model. Based on fatigue signals and a preset fatigue prediction model, a driving fatigue prediction result is obtained; wherein, the fatigue prediction model is a deep neural network; during prediction, abnormal fatigue signals corresponding to problems with abnormal driving prediction results within the same instant are removed. Utilize abnormal driving prediction results and fatigue prediction results to make safe driving judgments; Abnormal driving linear prediction model: in, It is a prediction result; It is the intercept term; It is the independent variable. =1, 2, 3…n, which correspond to the values ​​of independent variables for heart rate, respiratory rate, driving speed, driving duration and in-vehicle temperature, respectively. It is the independent variable Weighting coefficients; This is the random error term; The measured value of the independent variable; These are the normal reference values ​​for the independent variable; Adjust the parameters for the independent variable.

2. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in claim 1, characterized in that, The head, mouth, and eye contours in the image information are located and cropped; images showing nodding, yawning, closing eyes, and shaking the head are marked as abnormal images; among the abnormal images, if the abnormal driving prediction result at the corresponding time is yes, the abnormal image at the corresponding time is deleted.

3. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in claim 1, characterized in that, The error between the output of the deep neural network and the expected output is calculated, and then the error is backpropagated to the hidden layer to update the weights between neurons in the deep neural network. The weight updates are repeated until the performance of the deep neural network reaches a preset level or the parameters of the deep neural network converge.

4. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in claim 1, characterized in that, In the abnormal driving linear prediction model, the normal prediction result, along with facial expression image information, is used as the input to the fatigue prediction model.

5. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in claim 1, characterized in that, When the abnormal driving prediction result exceeds the preset value, the hazard lights will be turned on to issue an alarm, and the vehicle will be stopped and the gear will be switched to P during the alarm period. Fatigue prediction will not be performed. If the abnormal driving prediction result does not exceed the preset value, fatigue prediction is performed, and an audible alarm is triggered when the risk of fatigue driving occurs.

6. A safe driving judgment system based on in-vehicle microwave signals and fatigue signals, characterized in that, include: The data acquisition module is configured to acquire the driver's microwave signal, fatigue signal, driving time, and in-vehicle temperature. The abnormal driving prediction module is configured to: obtain abnormal driving prediction results based on the microwave signal, driving duration and in-vehicle temperature, and a preset abnormal driving linear prediction model; wherein, the abnormal driving linear prediction model is a generalized linear regression analysis model. The fatigue prediction module is configured to: obtain driving fatigue prediction results based on fatigue signals and a preset fatigue prediction model; wherein, the fatigue prediction model is a deep neural network; during prediction, abnormal fatigue signals corresponding to problems with abnormal driving prediction results within the same instant are eliminated; The safe driving judgment module is configured to make a safe driving judgment based on the abnormal driving prediction results and fatigue prediction results. Abnormal driving linear prediction model: in, It is a prediction result; It is the intercept term; It is the independent variable. =1, 2, 3…n, which correspond to the values ​​of independent variables for heart rate, respiratory rate, driving speed, driving duration and in-vehicle temperature, respectively. It is the independent variable Weighting coefficients; This is the random error term; The measured value of the independent variable; These are the normal reference values ​​for the independent variable; Adjust the parameters for the independent variable.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the program, it implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in any one of claims 1-5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in any one of claims 1-5.

Images