Safe driving judgment method and system based on in-vehicle microwave signal and fatigue signal

Abstract

The invention belongs to the technical field of active driving safety, and provides a safe driving judgment method and system based on an in-vehicle microwave signal and a fatigue signal, and the method comprises the steps: obtaining the microwave signal, the fatigue signal, the driving duration and the in-vehicle temperature of a driver; and performing safe driving judgment by using the abnormal driving prediction result and the fatigue prediction result. On the basis that abnormal driving prediction and fatigue prediction are carried out by means of a generalized linear regression analysis model and a deep neural network, microwave signals, fatigue signals, driving duration and in-vehicle temperature are comprehensively considered, the problem existing when alarm information is considered to be single is solved, prediction precision is improved, and meanwhile, in the fatigue prediction process, the fatigue prediction accuracy is improved. The corresponding abnormal fatigue signals when the abnormal driving prediction result goes wrong at the same moment are eliminated, the problem of control disorder caused by prediction according to the abnormal driving prediction result and the fatigue driving prediction result at the same time is avoided, and meanwhile the overall data processing amount of the model is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of active driving safety, and in particular relates to a safe driving judgment method and system based on microwave signals and fatigue signals in a vehicle. Background Art

[0002] The current fatigue driving reminder mainly uses an infrared camera facing the driver to monitor the driver's head, eyes, face and other details in real time, and then performs pattern recognition on the acquired information data to make a judgment on fatigue or distraction; the fatigue monitoring system (Driver Monitoring System, DMS) can detect the driver's fatigue driving and distraction, as well as unexpected situations where he cannot drive, and give the driver a warning reminder. The current in-car microwave radar is mainly used for rear-seat child monitoring. After the vehicle is turned off and the four doors and two lids are closed, the living thing monitoring is activated. When the characteristics of a child left behind are detected, the driver and surrounding personnel will be informed that a child has been left behind in the car through horns / double flashes / text message notifications, etc., and the driver's heart rate and breathing rate as well as fatigue level can be detected in real time.

[0003] Although current vehicles can monitor the fatigue information, heart rate and respiratory rate of occupants through infrared cameras and microwave signals, and make corresponding judgments and controls; however, after judging fatigue driving and sudden illness of the driver based on signals such as heart rate and respiratory rate, the alarm information is relatively simple and rigid, and the corresponding alarm, gear switching and automatic rescue call control strategies are also relatively simple, and cannot effectively divide the safety level, resulting in problems such as excessive corresponding control strategies or false alarms; and, when performing corresponding alarms and emergency controls, there is a lack of connection between the driver's fatigue and illness information and the driving environment, which increases the probability of accidents in the controlled vehicle. Summary of the invention

[0004] In order to solve the above problems, the present invention proposes a safe driving judgment method and system based on microwave signals and fatigue signals in the vehicle. The present invention uses a generalized linear regression analysis model and a deep neural network to perform abnormal driving prediction and fatigue prediction. By comprehensively considering microwave signals, fatigue signals, driving time and temperature in the vehicle, the problem existing when the alarm information is relatively single is solved, and the prediction accuracy is improved. At the same time, when predicting fatigue, the abnormal fatigue signal corresponding to the abnormal driving prediction result at the same time is eliminated, so as to avoid the problem of control confusion caused by predicting according to the abnormal driving prediction result and the fatigue driving prediction result at the same time, and reduce the overall data processing amount of the model.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0006] In a first aspect, the present invention provides a safe driving judgment method based on microwave signals and fatigue signals in a vehicle, comprising:

[0007] Obtain the driver's microwave signal, fatigue signal, driving time and temperature inside the car;

[0008] According to the microwave signal, the driving time and the temperature inside the vehicle, and a preset abnormal driving linear prediction model, an abnormal driving prediction result is obtained; wherein the abnormal driving linear prediction model is a generalized linear regression analysis model;

[0009] Determine the driving fatigue prediction result according to the fatigue signal and the preset fatigue prediction model; wherein the fatigue prediction model is a deep neural network; when predicting, eliminate the abnormal fatigue signal corresponding to the abnormal driving prediction result at the same time;

[0010] Utilize abnormal driving prediction results and fatigue prediction results to make safe driving judgments.

[0011] Further, abnormal driving linear prediction model:

[0012] y=β0+β1x1+β2x2+…+β n x n +ε

[0013]

[0014] Among them, y is the predicted result; β0 is the intercept term; x i is the independent variable, i = n, n = 1, 2, 3 and n, corresponding to the independent variable values ​​of heart rate, respiratory rate, driving speed, driving time and temperature inside the car respectively; β i is the independent variable x i The weight coefficient; ε is the random error term; X i is the measured value of the independent variable; X w is the normal reference value of the independent variable; γ i Adjust parameters for independent variables.

[0015] Furthermore, the head contour, mouth contour and eye contour in the image information are located and cropped; images with nodding, yawning, closing eyes and shaking heads are marked as abnormal images; among the abnormal images, if the abnormal driving prediction result at the corresponding time is yes, the abnormal image at the corresponding time is deleted.

[0016] Furthermore, the error between the deep neural network output and the expected output is calculated, and then the error is back-propagated to the hidden layer to update the weights between neurons in the deep neural network. The weights are updated repeatedly until the performance of the deep neural network reaches a preset level or the parameters of the deep neural network converge.

[0017] Furthermore, the normal prediction results of the abnormal driving linear prediction model are used together with the facial expression image information as the input of the fatigue prediction model.

[0018] Furthermore, when the abnormal driving prediction result exceeds the preset value, the hazard lights are turned on to sound an alarm and the vehicle stops and shifts to P gear during the alarm period; no fatigue prediction is performed; when the abnormal driving prediction result does not exceed the preset value, fatigue prediction is performed, and when there is a risk of fatigue driving, a sound alarm is issued.

[0019] In a second aspect, the present invention further provides a safe driving judgment system based on microwave signals and fatigue signals in a vehicle, comprising:

[0020] The data acquisition module is configured to: obtain the driver's microwave signal, fatigue signal, driving time and in-vehicle temperature;

[0021] The abnormal driving prediction module is configured to obtain an abnormal driving prediction result according to the microwave signal, the driving time and the temperature inside the vehicle, and a preset abnormal driving linear prediction model; wherein the abnormal driving linear prediction model is a generalized linear regression analysis model;

[0022] The fatigue prediction module is configured to: determine the driving fatigue prediction result according to the fatigue signal and the preset fatigue prediction model; wherein the fatigue prediction model is a deep neural network; when predicting, eliminate the abnormal fatigue signal corresponding to the abnormal driving prediction result when there is a problem at the same time;

[0023] The safe driving judgment module is configured to make a safe driving judgment using the abnormal driving prediction result and the fatigue prediction result.

[0024] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in the first aspect.

[0025] In a fourth aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein when the processor executes the program, the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in the first aspect are implemented.

[0026] In a fifth aspect, the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in the first aspect are implemented.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. In the present invention, based on the abnormal driving prediction and fatigue prediction with the help of generalized linear regression analysis model and deep neural network, the microwave signal, fatigue signal, driving time and temperature inside the vehicle are comprehensively considered to solve the problem existing when the alarm information is relatively single, and the prediction accuracy is improved. At the same time, when predicting fatigue, the abnormal fatigue signal corresponding to the abnormal driving prediction result at the same time is eliminated, so as to avoid the problem of control confusion caused by predicting according to the abnormal driving prediction result and the fatigue driving prediction result at the same time, and reduce the overall data processing amount of the model.

[0029] 2. In the present invention, during the training and prediction of the fatigue prediction model, the normal prediction results and the facial expression image information in the prediction results of the abnormal driving linear prediction model are used as the input of the fatigue prediction model, which is equivalent to taking into account the heart rate, breathing rate, driving speed, driving time and temperature inside the car, etc. during fatigue prediction, thereby improving the prediction accuracy; and when the abnormal driving prediction result exceeds the preset value, the double flash is turned on for alarm and the vehicle stops and the gear is switched to P gear during the alarm period; no fatigue prediction is performed; when the abnormal driving prediction result does not exceed the preset value, fatigue prediction is performed; when there is a risk of fatigue driving, a sound alarm is issued; the abnormal driving prediction result is used as a high-level prediction, and the fatigue prediction is used as a relatively low-level prediction, thereby realizing alarms and strategies based on safety level division. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0031] Figure 1 is a flow chart of the method of embodiment 1 of the present invention;

[0032] Figure 2 The time axis of Example 1 of the present invention;

[0033] Figure 3 Schematic diagram of a deep neural network according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0036] Embodiment 1:

[0037] This embodiment provides a safe driving judgment method based on in-vehicle microwave signals and fatigue signals. The driver's heart rate condition and fatigue level are tracked through in-vehicle microwave radar perception detection. Combined with the visual function of DMS (fatigue monitoring system), the driver's facial expression is recognized to judge the driver's health status. The judgment results of abnormal breathing rate and heartbeat data are output to the instrument warning, and then the vehicle is stopped and rescue is called for, thereby carrying out rescue and improving the driver's driving safety.

[0038] Optionally, the method is implemented based on a 66GHz in-car microwave radar (CIR), a fatigue detection system (DMS), an on-board instrument (ICM), a transmission control unit (TCU), a braking system (APB) and a central gateway (CGW). Among them, the TCU is responsible for sending the gear signal, the CIR is responsible for sending the driver's heart rate and breathing rate signal, and the DMS is responsible for sending the driver's fatigue signal; the CIR monitors and sends the driver's health status, and the DMS detects the driver's expression signal through vision. The CIR releases the driver's health status signal by combining the detected signal with the DMS signal. The ICM instrument displays the driver's heart rate and breathing rate and issues an alarm (double flash), the TCU is responsible for sending the gear signal, and the SOS is responsible for sending the rescue signal. CIR uses 60-64GHz to meet the frequency band requirements of most markets. Two radars need to be installed, and the installation positions are installed in the middle of the second row roof and the main driver's seat back, and the detection distance is ≥3m. Specifically, the method in this embodiment includes:

[0039] S1. Predict abnormal driving based on microwave signals:

[0040] Optionally, microwave radar is used to collect microwave information when the driver is driving normally as a normal data set, and microwave information when the driver has a malfunction due to illness is collected as an abnormal data set; the dependent variable is set to whether abnormal driving occurs, and the output is "yes" when abnormal driving occurs, otherwise it is "no", and the optional independent variables are heart rate, respiratory rate, driving speed, driving time and temperature inside the car. Then, through generalized linear regression analysis, the weights of the independent variables are obtained, and finally the possibility of abnormal driving during driving is predicted based on the obtained weights.

[0041] Specifically, because abnormal driving prediction is a binary result ("yes" / "no"), we can choose binomial distribution as the distribution family of the response variable, and we can use logistic regression; in the generalized linear regression model, the link function links the linear predictor to the expected value of the response variable. The link function uses the logit function to map the probability to all possible real values. Construct an abnormal driving linear prediction model:

[0042] y=β0+β1x1+β2x2+…+β n x n +ε

[0043]

[0044] Among them, y is the prediction result. When it is greater than the preset value, abnormal driving is predicted; β0 is the intercept term; x i is the independent variable, i = n, n = 1, 2, 3 and n, corresponding to the independent variable values ​​of heart rate, respiratory rate, driving speed, driving time and temperature inside the car respectively; β i is the independent variable x i The weight coefficient; ∈ is the random error term; X i is the measured value of the independent variable; X w is the normal reference value of the independent variable; γ i To adjust the parameters of the independent variables, the ratio of the measured values ​​to the normal reference values ​​is adjusted so that there is no difference in magnitude between the respective variables after adjustment.

[0045] Optionally, use the statistical R or Python statsmodels library to fit the model.

[0046] S2. Fatigue driving prediction based on fatigue signals:

[0047] S2.1. Optionally, an infrared camera is used to collect image information of the driver's facial expressions, including drowsiness, yawning, closing eyes, and shaking head.

[0048] S2.2. Feature extraction and screening of the collected expression signals; specifically, positioning and cropping the head contour, mouth contour and eye contour in the image information; marking images with nodding, yawning, closing eyes and shaking heads as abnormal images; in abnormal images, if the abnormal driving prediction result at the corresponding time is yes, then delete the abnormal image at the corresponding time, for example Figure 2 In the example, at time t1, if the abnormal driving prediction result y1 exceeds the preset safety value, it is considered that there is a problem with the driver's physical condition at this moment, and an alarm and corresponding control are required according to the abnormal driving prediction result, so as to avoid control confusion caused by predicting according to both the abnormal driving prediction result and the fatigue driving prediction result, and reduce the overall data processing amount of the model.

[0049] S2.3, normalize the expression signal data:

[0050]

[0051] Where a is the expression signal; a' represents the normalized expression signal a'∈[0,1]; a maxand a min It represents the maximum and minimum values ​​in the facial expression signal; when nodding and shaking the head, the maximum swing scale is used to represent the specific swing degree, and when yawning and closing the eyes, the frequency of occurrence within a certain period of time is used as the specific data to be processed.

[0052] S2.4. The labor prediction model uses a trained deep neural network (DNN), which includes an input layer, a hidden layer, and an output layer.

[0053] The deep neural network is positioned as: O∈R m ×R n , where the input of the model is the m-dimensional state vector a=a1, a2…a m , the output is an n-dimensional vector O(a) n ; The calculation between neurons is:

[0054]

[0055] in, is the activation function of the i-th neuron in layer l; Z ij is the action from the i-th neuron in layer I to the j-th neuron in layer I+1; is the weight matrix; is the basis function vector of neuron j; f() is the nonlinear softmax function as the activation function.

[0056] The weight w represents the degree of influence of the input signal on the target neuron. The connection formed by the weights between neurons constitutes the learning ability of the network, which enables it to fit the relationship between input and output by adjusting these weights through training. In this embodiment, the gradient descent algorithm is used to update the weight w of the model. Specifically, the error between the model output and the expected output is calculated, and then the error is back-propagated to the hidden layer to update the weight w. The weight w is repeatedly updated until the performance of the model reaches a satisfactory level or the parameters of the model have converged.

[0057] The output layer divisor result of the fatigue prediction model is whether there is a risk of fatigue driving or not.

[0058] When training and predicting the fatigue prediction model, the normal prediction result y and the facial expression image information in the prediction results of the abnormal driving linear prediction model are used as the input of the fatigue prediction model. This is equivalent to taking heart rate, breathing rate, driving speed, driving time and temperature inside the car into consideration during fatigue prediction, thereby improving the prediction accuracy.

[0059] S3. Use the abnormal driving prediction results and fatigue prediction results to make safe driving judgments:

[0060] When the abnormal driving prediction result y exceeds the preset value, the hazard lights are turned on to give an alarm and the vehicle stops and shifts to P gear during the alarm period, and a CAN signal is triggered to the E-CALL module to automatically call for help; no fatigue prediction is performed;

[0061] When the abnormal driving prediction result y does not exceed the preset value, fatigue prediction is performed; when there is a risk of fatigue driving, a sound alarm is issued.

[0062] Embodiment 2:

[0063] This embodiment provides a safe driving judgment system based on microwave signals and fatigue signals in a vehicle, including:

[0064] The data acquisition module is configured to: obtain the driver's microwave signal, fatigue signal, driving time and in-vehicle temperature;

[0065] The abnormal driving prediction module is configured to obtain an abnormal driving prediction result according to the microwave signal, the driving time and the temperature inside the vehicle, and a preset abnormal driving linear prediction model; wherein the abnormal driving linear prediction model is a generalized linear regression analysis model;

[0066] The fatigue prediction module is configured to: determine the driving fatigue prediction result according to the fatigue signal and the preset fatigue prediction model; wherein the fatigue prediction model is a deep neural network; when predicting, eliminate the abnormal fatigue signal corresponding to the abnormal driving prediction result when there is a problem at the same time;

[0067] The safe driving judgment module is configured to make a safe driving judgment using the abnormal driving prediction result and the fatigue prediction result.

[0068] The working method of the system is the same as the safe driving judgment method based on the microwave signal and fatigue signal in the vehicle in Example 1, and will not be repeated here.

[0069] Embodiment 3:

[0070] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the safe driving judgment method based on the in-vehicle microwave signal and fatigue signal described in Example 1 are implemented.

[0071] Embodiment 4:

[0072] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in Example 1 are implemented.

[0073] Embodiment 5:

[0074] This embodiment provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals described in Example 1 are implemented.

[0075] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. A safe driving judgment method based on microwave signals and fatigue signals in a vehicle, characterized in that: include: Obtain the driver's microwave signal, fatigue signal, driving time and temperature inside the car; According to the microwave signal, the driving time and the temperature inside the vehicle, and a preset abnormal driving linear prediction model, an abnormal driving prediction result is obtained; wherein the abnormal driving linear prediction model is a generalized linear regression analysis model; Determine the driving fatigue prediction result according to the fatigue signal and the preset fatigue prediction model; wherein the fatigue prediction model is a deep neural network; when predicting, eliminate the abnormal fatigue signal corresponding to the abnormal driving prediction result at the same time; Utilize abnormal driving prediction results and fatigue prediction results to make safe driving judgments.

2. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as claimed in claim 1, characterized in that: Abnormal driving linear prediction model: y=β0+β1x1+β2x2+…+β n x n +∈ Among them, y is the predicted result; β0 is the intercept term; x i is the independent variable, i = n, n = 1, 2, 3 and n, corresponding to the independent variable values ​​of heart rate, respiratory rate, driving speed, driving time and temperature inside the car respectively; β i is the independent variable x i The weight coefficient; ∈ is the random error term; X i is the measured value of the independent variable; X w is the normal reference value of the independent variable; i Adjust parameters for independent variables.

3. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as claimed in claim 1, characterized in that: The head contour, mouth contour and eye contour in the image information are located and cropped; images with nodding, yawning, closing eyes and shaking heads are marked as abnormal images; among the abnormal images, if the abnormal driving prediction result at the corresponding time is yes, the abnormal image at the corresponding time is deleted.

4. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as claimed in claim 1, characterized in that: The error between the deep neural network output and the expected output is calculated, and then the error is back-propagated to the hidden layer to update the weights between neurons in the deep neural network. The weights are updated repeatedly until the performance of the deep neural network reaches a preset level or the parameters of the deep neural network converge.

5. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as claimed in claim 1, characterized in that: The normal prediction results of the abnormal driving linear prediction model are taken together with the facial expression image information as the input of the fatigue prediction model.

6. The safe driving judgment method based on in-vehicle microwave signals and fatigue signals as claimed in claim 1, characterized in that: When the abnormal driving prediction result exceeds the preset value, the hazard lights will be turned on to warn you and the vehicle will stop and shift to P gear during the warning period. No fatigue prediction is performed; When the abnormal driving prediction result does not exceed the preset value, fatigue prediction is performed, and when there is a risk of fatigue driving, a sound alarm is issued.

7. A safe driving judgment system based on microwave signals and fatigue signals in a vehicle, characterized in that: include: The data acquisition module is configured to: obtain the driver's microwave signal, fatigue signal, driving time and in-vehicle temperature; The abnormal driving prediction module is configured to obtain an abnormal driving prediction result according to the microwave signal, the driving time and the temperature inside the vehicle, and a preset abnormal driving linear prediction model; wherein the abnormal driving linear prediction model is a generalized linear regression analysis model; The fatigue prediction module is configured to: determine the driving fatigue prediction result according to the fatigue signal and the preset fatigue prediction model; wherein the fatigue prediction model is a deep neural network; when predicting, eliminate the abnormal fatigue signal corresponding to the abnormal driving prediction result when there is a problem at the same time; The safe driving judgment module is configured to make a safe driving judgment using the abnormal driving prediction result and the fatigue prediction result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the safe driving judgment method based on in-vehicle microwave signals and fatigue signals as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor executes the program, the steps of the safe driving judgment method based on the in-vehicle microwave signal and fatigue signal as described in any one of claims 1-6 are implemented.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the safe driving judgment method based on the in-vehicle microwave signal and fatigue signal as described in any one of claims 1 to 6 are implemented.

Images