Vehicle warning method and system for identifying dangerous driving behavior of driver

Abstract

The present invention relates to the field of vehicle control, and proposes a vehicle early warning method and system for identifying dangerous driving behaviors of drivers. By real-time monitoring and identifying the driving behaviors, physiological and psychological states of the driver, a comprehensive data analysis module and a multimodal feature recognition network are designed to comprehensively analyze and judge the driving state and driving behaviors of the driver. At the same time, the method has a more accurate analysis capability for the driver's retaliatory driving due to personal emotions or uncontrolled driving due to physiological diseases, and further improves the accuracy of identifying dangerous driving behaviors, thereby timely and targeted active intervention is carried out, and dangerous driving alarms and dangerous driving feedback reports are carried out at the same time, so as to further improve the safety of active protection. The present invention improves the accuracy of identifying dangerous driving behaviors and the safety performance of vehicle early warning.

Description

Technical Field

[0001] The present invention relates to the field of vehicle control, and in particular to a vehicle early warning method and system for identifying dangerous driving behavior of a driver. Background Art

[0002] With the rapid development of the automobile industry, vehicle driving safety has gradually become an important evaluation indicator, especially the safety monitoring and safety protection of the driver's driving behavior. When the driver actively or passively makes dangerous driving behaviors, the vehicle can promptly warn and actively intervene or prevent the driver's dangerous driving behaviors, which is an important part of vehicle driving safety.

[0003] In the existing technology, active identification, warning and intervention of drivers' dangerous behaviors are often carried out by using computer vision technology to monitor and warn drivers of fatigue driving or non-focused driving behaviors (such as making phone calls, using electronic devices). However, when drivers drive in revenge due to personal emotions or drive out of control due to physiological diseases, existing methods for identifying drivers' dangerous driving behaviors are often difficult to judge, and the vehicle cannot issue warnings or actively intervene in time.

[0004] Therefore, how to design a vehicle warning method for identifying dangerous driving behaviors of drivers in order to accurately and comprehensively identify dangerous driving behaviors of drivers is an urgent problem to be solved. Summary of the invention

[0005] Based on this, the present invention proposes a vehicle warning method and system for identifying dangerous driving behaviors of drivers. By monitoring and identifying the driver's driving behavior, physiological and psychological state in real time, a comprehensive data analysis module and a multimodal feature recognition network are designed to comprehensively analyze and judge the driver's driving state and driving behavior. At the same time, it has more accurate analysis capabilities for the driver's retaliatory driving due to personal emotions or out-of-control driving due to physiological diseases, further improving the accuracy of identifying dangerous driving behaviors, thereby timely and targeted active intervention, and at the same time performing dangerous driving alarms and dangerous driving feedback reports to further improve the safety of active protection. The present invention improves the accuracy of identifying dangerous driving behaviors and the safety performance of vehicle warnings.

[0006] The present invention proposes a vehicle early warning method for identifying dangerous driving behavior of a driver, comprising:

[0007] Collecting vehicle driving state data and driver driving state data, wherein the vehicle driving state data includes vehicle driving radar data and vehicle driving controller data, and the driver driving state data includes driver physiological state data and driver emotional state data;

[0008] The driver's dangerous driving behavior is judged according to the vehicle driving state data and the driver's driving state data, wherein the driver's dangerous driving behavior judgment is based on a comprehensive data analysis module and a multimodal feature recognition network, wherein the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork, and the multimodal features include facial image features, voice audio features, and voice text features of the driver;

[0009] If it is determined that the driver is in a dangerous driving state, an active intervention operation is performed according to the vehicle driving state data, and then a dangerous driving alarm is issued and a dangerous driving feedback report is made according to the driver's driving state data. The active intervention operation is performed according to the vehicle driving radar data.

[0010] In summary, according to the above-mentioned vehicle warning method for identifying dangerous driving behaviors of drivers, a comprehensive data analysis module and a multimodal feature recognition network are designed by monitoring and identifying the driver's driving behavior, physiological and psychological state in real time, to comprehensively analyze and judge the driver's driving state and driving behavior. At the same time, it has more accurate analysis capabilities for the driver's retaliatory driving due to personal emotions or out-of-control driving due to physiological diseases, further improving the accuracy of identifying dangerous driving behaviors, thereby timely and targeted active intervention, and at the same time, performing dangerous driving alarms and dangerous driving feedback reports to further improve the safety of active protection. The present invention improves the accuracy of identifying dangerous driving behaviors and the safety performance of vehicle warnings. Specifically, the vehicle driving state data and the driver driving state data are collected, the vehicle driving state data includes the vehicle driving radar data and the vehicle driving controller data, the driver driving state data includes the driver physiological state data and the driver emotional state data, the driver's driving behavior, physiological and psychological state are monitored in real time, and the existence of abnormal behavior and state is discovered in time, and the driver's dangerous driving behavior is judged according to the vehicle driving state data and the driver driving state data. The driver's dangerous driving behavior judgment is based on a comprehensive data analysis module and a multimodal feature recognition network, and the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork. The multimodal features include facial image features, voice audio features and voice text features of the driver, which further improve the accuracy of dangerous behavior recognition and timely carry out subsequent active intervention to avoid the risk of safety accidents expanding. If the driver is determined to be in a dangerous driving state, an active intervention operation is performed according to the vehicle driving state data, and then a dangerous driving alarm is issued and a dangerous driving feedback report is performed according to the driver driving state data. The active intervention operation is performed according to the vehicle driving radar data, and active intervention is carried out in a targeted manner, thereby improving the safety protection performance of the overall early warning method. The present invention improves the accuracy of dangerous driving behavior recognition and the safety performance of vehicle early warning.

[0011] Furthermore, the comprehensive data analysis module specifically includes:

[0012] After collecting the vehicle driving status data and the driver driving status data in real time;

[0013] Obtaining anomaly coefficients based on a multi-view anomaly detection algorithm;

[0014] If the abnormal coefficient is in the first abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are good, and the driver is in a low-risk stage. The probability of implementing retaliatory driving behavior is low, and the current judgment cycle is ended. represents a low-risk abnormal coefficient threshold, where the low risk means that the probability of retaliatory driving behavior is less than or equal to 10%;

[0015] If the abnormal coefficient is in the second abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and in a general risk stage, and the probability of implementing retaliatory driving behavior is general. The multimodal data of the current driver is collected and input into the multimodal feature recognition network as a basis for judgment, and a multimodal judgment is performed. represents the medium risk abnormal coefficient threshold, where the medium risk means that the probability of retaliatory driving behavior is less than or equal to 50% and greater than 10%;

[0016] If the abnormal coefficient is in the third abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and at a high risk stage, and the probability of implementing retaliatory driving behavior is high. The dangerous state feedback is provided according to the communication transmission module, and the voice output device and image acquisition device in the car are connected to monitor and actively question the driver, and the retaliatory driving behavior is judged according to the risk score algorithm. represents a high-risk abnormal coefficient threshold, where the high risk means that the probability of retaliatory driving behavior is less than or equal to 90% and greater than 50%;

[0017] If the abnormal coefficient is in the fourth abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and at a major risk stage, and the probability of retaliatory driving behavior is very high. The dangerous state feedback is performed according to the communication transmission module and active intervention operations are initiated.

[0018] Furthermore, the multi-view anomaly detection algorithm and the risk scoring algorithm specifically include:

[0019] The multi-view anomaly detection algorithm includes a local anomaly detection algorithm and a global anomaly detection algorithm;

[0020] The local anomaly detection scores of all indicators in the vehicle driving state data and the driver driving state data at the current moment are calculated according to the local anomaly detection algorithm. The specific algorithm of the local anomaly detection algorithm is as follows:

[0021] ,

[0022] in, represents the local anomaly detection score, Represents data points in vehicle driving status data and driver driving status data, express The neighbor set of express The number of neighbors of express The neighboring data points of express and The Euclidean distance of

[0023] The global anomaly detection score of all indicators in the vehicle driving state data and the driver driving state data at the current moment is calculated according to the global anomaly detection algorithm. The specific algorithm of the global anomaly detection algorithm is as follows:

[0024]

[0025] in, represents the global anomaly detection score, Indicators representing vehicle driving status data and driver driving status data, Indicates the total number of indicators, express The probability density of

[0026] The anomaly coefficient is calculated according to the local anomaly detection score and the global anomaly detection score. The specific algorithm for calculating the anomaly coefficient is as follows:

[0027]

[0028] in, represents the anomaly coefficient, Indicates normalization processing;

[0029] The specific formula of the risk score algorithm is as follows:

[0030] ,

[0031] ,

[0032] in, , , They represent the driver status index, vehicle status index, and environmental factor index respectively. , , , Respectively represent different indicator risk ranges, It means that the driver is in good condition, the vehicle is in good condition, and there are few pedestrians and obstacles in the environment. The few pedestrians and obstacles in the environment means that there are few pedestrians and obstacles in the vehicle radar range. , It indicates that the driver is in general condition, the vehicle is in general condition, and there are few pedestrians and obstacles in the environment. The few pedestrians and obstacles in the environment means that there are few pedestrians and obstacles in the vehicle radar range. , It indicates that the driver is in a bad state, the vehicle is in a bad state, and there are many pedestrians and obstacles in the environment. The number of pedestrians and obstacles in the environment indicates that there are many pedestrians and obstacles in the vehicle radar range. , It means that the driver is in a very bad state, the vehicle is in a very bad state, and there are dense pedestrians and obstacles in the environment. The dense pedestrians and obstacles in the environment means that there are dense pedestrians and obstacles in the vehicle radar range. The good, average, poor and extremely poor driver and vehicle states correspond to different fluctuation levels of the driver state index and the vehicle state index, respectively. Represents the single risk score output score of the indicator data, Represents the total risk score.

[0033] Furthermore, the multimodal feature extraction subnetwork specifically includes:

[0034] According to the steering wheel touch pressure sensor, a steering wheel touch pressure image of the driver at the current moment is obtained;

[0035] According to the in-vehicle image collector, the driver's facial information image at the current moment is obtained;

[0036] According to the in-car audio collector, the driver's voice information at the current moment is obtained;

[0037] Performing image registration processing on the steering wheel touch and pressure image and the driver's facial information image, and then performing image synthesis, wherein the image synthesis is to perform wavelet decomposition on the steering wheel touch and pressure image and the driver's facial information image based on wavelet transform to obtain a plurality of sub-images in different frequency bands, and fuse them to generate a synthesized image;

[0038] Performing an image feature extraction operation on the synthesized image according to a convolutional neural sub-network to obtain basic image features of the driver, and then performing feature enhancement on the basic image features of the driver according to a local attention enhancement module to obtain enhanced image features of the driver;

[0039] Performing an audio feature extraction operation on the driver's voice information according to the Mel-frequency cepstral coefficient to obtain the driver's basic audio features, and then performing feature enhancement on the driver's basic audio features according to the key voice attention enhancement module to obtain the driver's enhanced audio features;

[0040] The driver's text information is generated according to the conversion of the driver's voice information, and then the text feature extraction operation is performed on the driver's voice information according to the global vector word embedding to obtain the driver's basic text features, and then the driver's basic text features are enhanced according to the keyword attention enhancement module to obtain the driver's enhanced text features.

[0041] Furthermore, the multimodal feature fusion sub-network specifically includes:

[0042] The multimodal feature fusion sub-network includes a multi-layer long short-term memory network, and the multi-layer long short-term memory network includes three network layers;

[0043] Inputting the driver enhancement image features obtained by the multimodal feature extraction subnetwork into the first network layer to obtain the first hidden layer state features of each neuron in the first network layer;

[0044] splicing the driver enhanced audio features obtained by the multimodal feature extraction subnetwork and the first hidden layer state features and inputting them into the second network layer to obtain the second hidden layer state features of each neuron in the second network layer;

[0045] The driver enhanced text features obtained by the multimodal feature extraction sub-network and the second hidden layer state features are concatenated and input into the third network layer to obtain the third hidden layer state features;

[0046] A multimodal fusion feature is obtained according to the state feature of the third hidden layer, and then the driver's dangerous driving behavior is judged according to the multimodal fusion feature to obtain a classification prediction result of the driver's dangerous driving behavior judgment.

[0047] Furthermore, if it is determined that the driver is in a dangerous driving state, the step of performing active intervention operations according to the vehicle driving state data specifically includes:

[0048] Actively intervene according to the vehicle driving radar data in the vehicle driving state data, wherein the active intervention is performed according to a preset active intervention strategy;

[0049] The specific algorithm of the preset active intervention strategy is as follows:

[0050] ,

[0051] in, Indicates the preset active intervention strategy, Indicates the current distance between the vehicle and the vehicle, pedestrian or obstacle in front. Indicates the preset first radar distance threshold, Indicates the preset second radar distance threshold;

[0052] When the active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle ahead is less than the preset first radar distance threshold, the vehicle deceleration device is controlled to decelerate the vehicle;

[0053] When active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle ahead is less than the preset second radar distance threshold at the current moment, the vehicle emergency brake is controlled to make the vehicle emergency brake to stop;

[0054] When active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle ahead is less than the preset first radar distance threshold or the preset second radar distance threshold, each tire of the vehicle is individually controlled to maintain vehicle stability;

[0055] When active intervention determines that the vehicle is in collision at the current moment, the airbag device in the vehicle is activated to perform active protection inside the vehicle, the engine hood is controlled to be lifted and the pedestrian protection airbag is actively opened to perform active protection outside the vehicle.

[0056] Furthermore, the step of providing a dangerous driving warning and providing a dangerous driving feedback report based on the driver's driving status data specifically includes:

[0057] When it is determined that the driver is in a dangerous driving state, the vehicle lighting equipment is controlled to flash the high and low beam lights alternately and turn on the hazard warning lights;

[0058] Then control the vehicle's external loudspeaker and vehicle warning horn to broadcast danger avoidance warnings and continuous short whistle sounds;

[0059] Based on the driver's dangerous driving status information, the driver's physiological status data and the driver's emotional status data, a dangerous driving feedback report is generated and communicated back to the cloud alarm system, and an alarm message is automatically sent to the preset emergency targets.

[0060] The present invention proposes a vehicle warning system for identifying dangerous driving behaviors of a driver, comprising:

[0061] A collection module, used for collecting vehicle driving state data and driver driving state data, wherein the vehicle driving state data includes vehicle driving radar data and vehicle driving controller data, and the driver driving state data includes driver physiological state data and driver emotional state data;

[0062] An analysis and judgment module, used for judging the driver's dangerous driving behavior according to the vehicle driving state data and the driver's driving state data, wherein the judgment of the driver's dangerous driving behavior is based on a comprehensive data analysis module and a multimodal feature recognition network, wherein the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork, and the multimodal features include facial image features, voice audio features, and voice text features of the driver;

[0063] The early warning intervention module is used to perform active intervention operations according to the vehicle driving status data if it is determined that the driver is in a dangerous driving state, and then issue a dangerous driving alarm and a dangerous driving feedback report according to the driver's driving status data. The active intervention operation is performed based on the vehicle driving radar data.

[0064] The present invention also provides a storage medium, which stores one or more programs, and when the programs are executed by a processor, the vehicle warning method for identifying dangerous driving behaviors of a driver as described above is implemented.

[0065] The present invention also provides a computer device, the computer device comprising a memory and a processor, wherein:

[0066] The memory is used to store computer programs;

[0067] When the processor is used to execute the computer program stored in the memory, the vehicle warning method for identifying dangerous driving behavior of the driver as described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 A flowchart of a vehicle warning method for identifying dangerous driving behavior of a driver proposed in the first embodiment of the present invention;

[0069] Figure 2 A flow chart of a vehicle warning method for identifying dangerous driving behavior of a driver proposed in a second embodiment of the present invention;

[0070] Figure 3 This is a schematic diagram of the structure of a vehicle warning system for identifying dangerous driving behaviors of a driver proposed in the third embodiment of the present invention.

[0071] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0072] In order to facilitate understanding of the present invention, the present invention will be described more fully below with reference to the relevant drawings. Several embodiments of the present invention are provided in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0073] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0075] See also Figure 1 , which is a flow chart of a vehicle early warning method for identifying dangerous driving behaviors of a driver proposed in a first embodiment of the present invention, wherein the vehicle early warning method for identifying dangerous driving behaviors of a driver comprises steps S01 to S03, wherein:

[0076] Step S01: collecting vehicle driving status data and driver driving status data;

[0077] It should be noted that, in this embodiment, the vehicle driving state data includes vehicle driving radar data and vehicle driving controller data, and the driver driving state data includes driver physiological state data and driver emotional state data.

[0078] Step S02: judging the driver's dangerous driving behavior based on the vehicle driving state data and the driver's driving state data;

[0079] It should be noted that in this embodiment, the driver's dangerous driving behavior judgment is based on a comprehensive data analysis module and a multimodal feature recognition network. The multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork. The multimodal features include the driver's facial image features, voice audio features, and voice text features. The comprehensive data analysis module specifically includes:

[0080] After collecting the vehicle driving status data and the driver driving status data in real time;

[0081] Obtaining anomaly coefficients based on a multi-view anomaly detection algorithm;

[0082] If the abnormal coefficient is in the first abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are good, and the driver is in a low-risk stage. The probability of implementing retaliatory driving behavior is low, and the current judgment cycle is ended. represents a low-risk abnormal coefficient threshold, where the low risk means that the probability of retaliatory driving behavior is less than or equal to 10%;

[0083] If the abnormal coefficient is in the second abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and in a general risk stage, and the probability of implementing retaliatory driving behavior is general. The multimodal data of the current driver is collected and input into the multimodal feature recognition network as a basis for judgment, and a multimodal judgment is performed. represents the medium risk abnormal coefficient threshold, where the medium risk means that the probability of retaliatory driving behavior is less than or equal to 50% and greater than 10%;

[0084] If the abnormal coefficient is in the third abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and at a high risk stage, and the probability of implementing retaliatory driving behavior is high. The dangerous state feedback is provided according to the communication transmission module, and the voice output device and image acquisition device in the car are connected to monitor and actively question the driver, and the retaliatory driving behavior is judged according to the risk score algorithm. represents a high-risk abnormal coefficient threshold, where the high risk means that the probability of retaliatory driving behavior is less than or equal to 90% and greater than 50%;

[0085] If the abnormal coefficient is in the fourth abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and at a major risk stage, and the probability of implementing retaliatory driving behavior is very high. The dangerous state feedback is performed according to the communication transmission module and active intervention operation is initiated;

[0086] The multi-view anomaly detection algorithm and the risk scoring algorithm specifically include:

[0087] The multi-view anomaly detection algorithm includes a local anomaly detection algorithm and a global anomaly detection algorithm;

[0088] The local anomaly detection scores of all indicators in the vehicle driving state data and the driver driving state data at the current moment are calculated according to the local anomaly detection algorithm. The specific algorithm of the local anomaly detection algorithm is as follows:

[0089] ,

[0090] in, represents the local anomaly detection score, Represents data points in vehicle driving status data and driver driving status data, express The neighbor set of express The number of neighbors of express The neighboring data points of express and The Euclidean distance of

[0091] The global anomaly detection score of all indicators in the vehicle driving state data and the driver driving state data at the current moment is calculated according to the global anomaly detection algorithm. The specific algorithm of the global anomaly detection algorithm is as follows:

[0092]

[0093] in, represents the global anomaly detection score, Indicators representing vehicle driving status data and driver driving status data, Indicates the total number of indicators, express The probability density of

[0094] The anomaly coefficient is calculated according to the local anomaly detection score and the global anomaly detection score. The specific algorithm for calculating the anomaly coefficient is as follows:

[0095]

[0096] in, represents the anomaly coefficient, Indicates normalization processing;

[0097] The specific formula of the risk score algorithm is as follows:

[0098] ,

[0099] ,

[0100] in, , , They represent the driver status index, vehicle status index, and environmental factor index respectively. , , , Respectively represent different indicator risk ranges, It means that the driver is in good condition, the vehicle is in good condition, and there are few pedestrians and obstacles in the environment. The few pedestrians and obstacles in the environment means that there are few pedestrians and obstacles in the vehicle radar range. , It indicates that the driver is in general condition, the vehicle is in general condition, and there are few pedestrians and obstacles in the environment. The few pedestrians and obstacles in the environment means that there are few pedestrians and obstacles in the vehicle radar range. , It indicates that the driver is in a bad state, the vehicle is in a bad state, and there are many pedestrians and obstacles in the environment. The number of pedestrians and obstacles in the environment indicates that there are many pedestrians and obstacles in the vehicle radar range. , It means that the driver is in a very bad state, the vehicle is in a very bad state, and there are dense pedestrians and obstacles in the environment. The dense pedestrians and obstacles in the environment means that there are dense pedestrians and obstacles in the vehicle radar range. The good, average, poor and extremely poor driver and vehicle states correspond to different fluctuation levels of the driver state index and the vehicle state index, respectively. Represents the single risk score output score of the indicator data, represents the total risk score;

[0101] The multimodal feature extraction subnetwork specifically includes:

[0102] According to the steering wheel touch pressure sensor, a steering wheel touch pressure image of the driver at the current moment is obtained;

[0103] According to the in-vehicle image collector, the driver's facial information image at the current moment is obtained;

[0104] According to the in-car audio collector, the driver's voice information at the current moment is obtained;

[0105] Performing image registration processing on the steering wheel touch and pressure image and the driver's facial information image, and then performing image synthesis, wherein the image synthesis is to perform wavelet decomposition on the steering wheel touch and pressure image and the driver's facial information image based on wavelet transform to obtain a plurality of sub-images in different frequency bands, and fuse them to generate a synthesized image;

[0106] Performing an image feature extraction operation on the synthesized image according to a convolutional neural sub-network to obtain basic image features of the driver, and then performing feature enhancement on the basic image features of the driver according to a local attention enhancement module to obtain enhanced image features of the driver;

[0107] Performing an audio feature extraction operation on the driver's voice information according to the Mel-frequency cepstral coefficient to obtain the driver's basic audio features, and then performing feature enhancement on the driver's basic audio features according to the key voice attention enhancement module to obtain the driver's enhanced audio features;

[0108] Convert the driver's voice information to generate driver text information, perform text feature extraction on the driver's voice information according to global vector word embedding to obtain driver basic text features, and perform feature enhancement on the driver's basic text features according to a keyword attention enhancement module to obtain driver enhanced text features;

[0109] The multimodal feature fusion sub-network specifically includes:

[0110] The multimodal feature fusion sub-network includes a multi-layer long short-term memory network, and the multi-layer long short-term memory network includes three network layers;

[0111] Inputting the driver enhancement image features obtained by the multimodal feature extraction subnetwork into the first network layer to obtain the first hidden layer state features of each neuron in the first network layer;

[0112] splicing the driver enhanced audio features obtained by the multimodal feature extraction subnetwork and the first hidden layer state features and inputting them into the second network layer to obtain the second hidden layer state features of each neuron in the second network layer;

[0113] The driver enhanced text features obtained by the multimodal feature extraction sub-network and the second hidden layer state features are concatenated and input into the third network layer to obtain the third hidden layer state features;

[0114] A multimodal fusion feature is obtained according to the state feature of the third hidden layer, and then the driver's dangerous driving behavior is judged according to the multimodal fusion feature to obtain a classification prediction result of the driver's dangerous driving behavior judgment.

[0115] Step S03: If it is determined that the driver is in a dangerous driving state, an active intervention operation is performed according to the vehicle driving state data, and then a dangerous driving alarm is issued and a dangerous driving feedback report is made according to the driver's driving state data;

[0116] It should be noted that in this embodiment, the active intervention operation is performed according to the vehicle driving radar data, and the active intervention is performed according to the vehicle driving radar data in the vehicle driving state data, and the active intervention is performed according to a preset active intervention strategy;

[0117] The specific algorithm of the preset active intervention strategy is as follows:

[0118] ,

[0119] in, Indicates the preset active intervention strategy, Indicates the current distance between the vehicle and the vehicle, pedestrian or obstacle in front. Indicates the preset first radar distance threshold, Indicates the preset second radar distance threshold;

[0120] When the active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle ahead is less than the preset first radar distance threshold, the vehicle deceleration device is controlled to decelerate the vehicle;

[0121] When active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle ahead is less than the preset second radar distance threshold at the current moment, the vehicle emergency brake is controlled to make the vehicle emergency brake to stop;

[0122] When active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle ahead is less than the preset first radar distance threshold or the preset second radar distance threshold, each tire of the vehicle is individually controlled to maintain vehicle stability;

[0123] When active intervention determines that the vehicle is in collision at the current moment, the airbag device in the vehicle is activated for active protection inside the vehicle, the engine hood is controlled to rise and the pedestrian protection airbag is actively activated for active protection outside the vehicle;

[0124] When it is determined that the driver is in a dangerous driving state, the vehicle lighting equipment is controlled to flash the high and low beam lights alternately and turn on the hazard warning lights;

[0125] Then control the vehicle's external loudspeaker and vehicle warning horn to broadcast danger avoidance warnings and continuous short whistle sounds;

[0126] Based on the driver's dangerous driving status information, the driver's physiological status data and the driver's emotional status data, a dangerous driving feedback report is generated and communicated back to the cloud alarm system, and an alarm message is automatically sent to the preset emergency targets.

[0127] In summary, according to the above-mentioned vehicle warning method for identifying dangerous driving behaviors of drivers, a comprehensive data analysis module and a multimodal feature recognition network are designed by monitoring and identifying the driver's driving behavior, physiological and psychological state in real time, to comprehensively analyze and judge the driver's driving state and driving behavior. At the same time, it has more accurate analysis capabilities for the driver's retaliatory driving due to personal emotions or out-of-control driving due to physiological diseases, further improving the accuracy of identifying dangerous driving behaviors, thereby timely and targeted active intervention, and at the same time, performing dangerous driving alarms and dangerous driving feedback reports to further improve the safety of active protection. The present invention improves the accuracy of identifying dangerous driving behaviors and the safety performance of vehicle warnings. Specifically, the vehicle driving state data and the driver driving state data are collected, the vehicle driving state data includes the vehicle driving radar data and the vehicle driving controller data, the driver driving state data includes the driver physiological state data and the driver emotional state data, the driver's driving behavior, physiological and psychological state are monitored in real time, and the existence of abnormal behavior and state is discovered in time, and the driver's dangerous driving behavior is judged according to the vehicle driving state data and the driver driving state data. The driver's dangerous driving behavior judgment is based on a comprehensive data analysis module and a multimodal feature recognition network, and the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork. The multimodal features include facial image features, voice audio features and voice text features of the driver, which further improve the accuracy of dangerous behavior recognition and timely carry out subsequent active intervention to avoid the risk of safety accidents expanding. If the driver is determined to be in a dangerous driving state, an active intervention operation is performed according to the vehicle driving state data, and then a dangerous driving alarm is issued and a dangerous driving feedback report is performed according to the driver driving state data. The active intervention operation is performed according to the vehicle driving radar data, and active intervention is carried out in a targeted manner, thereby improving the safety protection performance of the overall early warning method. The present invention improves the accuracy of dangerous driving behavior recognition and the safety performance of vehicle early warning.

[0128] See also Figure 2 , which is a flow chart of a vehicle warning method for identifying dangerous driving behaviors of a driver proposed in a second embodiment of the present invention, wherein the vehicle warning method for identifying dangerous driving behaviors of a driver comprises steps S11 to S15, wherein:

[0129] Step S11: After collecting the vehicle driving state data and the driver driving state data in real time, the abnormal coefficient is obtained according to the multi-view abnormality detection algorithm, and the abnormal interval of the abnormal coefficient is determined to determine the risk stage of the driver's driving behavior, physiological and psychological state, and determine the probability of the driver's retaliatory driving behavior for corresponding processing;

[0130] Step S12: obtaining a steering wheel touch and pressure image, a facial information image, and a driver's voice information of the driver at the current moment, performing image registration processing on the steering wheel touch and pressure image and the driver's facial information image, and then performing image synthesis, performing image feature extraction and feature enhancement on the synthesized image according to a convolutional neural subnetwork to obtain driver enhanced image features, performing audio feature extraction and feature enhancement on the driver's voice information according to the Mel cepstral coefficient to obtain driver enhanced audio features, generating driver text information according to the driver voice information conversion, and then performing text feature extraction and feature enhancement on the driver's voice information according to the global vector word embedding to obtain driver enhanced text features;

[0131] Step S13: inputting the driver enhanced image features into the first network layer of the multimodal feature fusion subnetwork to obtain the first hidden layer state features, splicing the driver enhanced audio features and the first hidden layer state features into the second network layer to obtain the second hidden layer state features, splicing the driver enhanced text features and the second hidden layer state features into the third network layer to obtain multimodal fusion features, and then performing the driver's dangerous driving behavior judgment to obtain the driver's dangerous driving behavior judgment classification prediction results;

[0132] Step S14: Actively intervene according to the vehicle driving radar data in the vehicle driving status data. When the active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle in front at the current moment is less than the preset first radar distance threshold, control the vehicle deceleration device to decelerate the vehicle. When the active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle in front at the current moment is less than the preset second radar distance threshold, control the vehicle emergency brake to make the vehicle emergency brake to stop. When the active intervention determines that the distance between the vehicle and the vehicle, pedestrian or obstacle in front at the current moment is less than the preset first radar distance threshold or the preset second radar distance threshold, each tire of the vehicle is individually controlled to maintain vehicle body stability. When the active intervention determines that the vehicle collides at the current moment, activate the in-vehicle airbag device to perform active in-vehicle protection, control the engine hood to lift and actively activate the pedestrian protection airbag to perform active outside-vehicle protection.

[0133] It should be noted that the first radar distance threshold in this embodiment is the minimum radar distance value of the current vehicle's rated directional cruise, so that after active intervention, the vehicle can enter the directional cruise state and keep following the vehicle to avoid the expansion of safety accidents caused by emergency stops. The second radar distance threshold in this embodiment is the shortest braking distance at the current vehicle speed, so that after active intervention, the vehicle can brake to a stop and avoid collisions.

[0134] Step S15: When it is determined that the driver is in a dangerous driving state, the vehicle lighting equipment is controlled to flash the high and low beams alternately and turn on the hazard warning lights. The vehicle's external speaker equipment and vehicle horn are then controlled to broadcast danger avoidance warnings and continuously sound the horn for short warnings. Based on the driver's dangerous driving state information, the driver's physiological state data, and the driver's emotional state data, a dangerous driving feedback report is generated and communicated to the cloud alarm system, and an alarm message is automatically sent to the preset emergency target.

[0135] In summary, according to the above-mentioned vehicle warning method for identifying dangerous driving behaviors of drivers, a comprehensive data analysis module and a multimodal feature recognition network are designed by monitoring and identifying the driver's driving behavior, physiological and psychological state in real time, to comprehensively analyze and judge the driver's driving state and driving behavior. At the same time, it has more accurate analysis capabilities for the driver's retaliatory driving due to personal emotions or out-of-control driving due to physiological diseases, further improving the accuracy of identifying dangerous driving behaviors, thereby timely and targeted active intervention, and at the same time, performing dangerous driving alarms and dangerous driving feedback reports to further improve the safety of active protection. The present invention improves the accuracy of identifying dangerous driving behaviors and the safety performance of vehicle warnings. Specifically, the vehicle driving state data and the driver driving state data are collected, the vehicle driving state data includes the vehicle driving radar data and the vehicle driving controller data, the driver driving state data includes the driver physiological state data and the driver emotional state data, the driver's driving behavior, physiological and psychological state are monitored in real time, and the existence of abnormal behavior and state is discovered in time, and the driver's dangerous driving behavior is judged according to the vehicle driving state data and the driver driving state data. The driver's dangerous driving behavior judgment is based on a comprehensive data analysis module and a multimodal feature recognition network, and the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork. The multimodal features include facial image features, voice audio features and voice text features of the driver, which further improve the accuracy of dangerous behavior recognition and timely carry out subsequent active intervention to avoid the risk of safety accidents expanding. If the driver is determined to be in a dangerous driving state, an active intervention operation is performed according to the vehicle driving state data, and then a dangerous driving alarm is issued and a dangerous driving feedback report is performed according to the driver driving state data. The active intervention operation is performed according to the vehicle driving radar data, and active intervention is carried out in a targeted manner, thereby improving the safety protection performance of the overall early warning method. The present invention improves the accuracy of dangerous driving behavior recognition and the safety performance of vehicle early warning.

[0136] See also Figure 3, which is a schematic diagram of the structure of a vehicle warning system for identifying dangerous driving behaviors of a driver proposed in a third embodiment of the present invention, and the system comprises:

[0137] The acquisition module 10 is used to acquire vehicle driving state data and driver driving state data, wherein the vehicle driving state data includes vehicle driving radar data and vehicle driving controller data, and the driver driving state data includes driver physiological state data and driver emotional state data;

[0138] An analysis and judgment module 20, configured to judge the driver's dangerous driving behavior based on the vehicle driving state data and the driver's driving state data, wherein the driver's dangerous driving behavior judgment is based on a comprehensive data analysis module and a multimodal feature recognition network, wherein the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork, and the multimodal features include facial image features, voice audio features, and voice text features of the driver;

[0139] The early warning intervention module 30 is used to perform active intervention operations according to the vehicle driving status data if it is determined that the driver is in a dangerous driving state, and then issue a dangerous driving alarm and a dangerous driving feedback report according to the driver's driving status data. The active intervention operation is performed based on the vehicle driving radar data.

[0140] The present invention also proposes a computer storage medium on which one or more programs are stored, and when the program is executed by a processor, the vehicle warning method for identifying dangerous driving behaviors of the driver is implemented.

[0141] The present invention also proposes a computer device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the above-mentioned vehicle warning method for identifying dangerous driving behavior of the driver.

[0142] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, "computer-readable medium" may be any device that can contain storage, communication, propagation or transmission of a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0143] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0144] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0145] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0146] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A vehicle warning method for identifying dangerous driving behavior of a driver, characterized in that: include: Collecting vehicle driving state data and driver driving state data, wherein the vehicle driving state data includes vehicle driving radar data and vehicle driving controller data, and the driver driving state data includes driver physiological state data and driver emotional state data; Performing a judgment on the driver's dangerous driving behavior based on the vehicle driving state data and the driver's driving state data, wherein the judgment on the driver's dangerous driving behavior is performed based on a comprehensive data analysis module and a multimodal feature recognition network, wherein the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork, and the multimodal features include facial image features, voice audio features, and voice text features of the driver; The comprehensive data analysis module specifically includes: After collecting the vehicle driving status data and the driver driving status data in real time; Obtain anomaly coefficients based on a multi-view anomaly detection algorithm; If the abnormal coefficient is in the first abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are good, and the driver is in a low-risk stage. The probability of implementing retaliatory driving behavior is small, and the current judgment cycle ends. represents a low-risk abnormal coefficient threshold, where low risk means the probability of retaliatory driving behavior is less than or equal to 10%; If the abnormal coefficient is in the second abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and in a general risk stage, and the probability of implementing retaliatory driving behavior is general. The multimodal data of the current driver is collected and input into the multimodal feature recognition network as a basis for judgment, and a multimodal judgment is performed. represents the medium-risk abnormal coefficient threshold, where the medium risk means that the probability of retaliatory driving behavior is less than or equal to 50% and greater than 10%; If the abnormal coefficient is in the third abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and at a high-risk stage, and the probability of implementing retaliatory driving behavior is high. The dangerous state feedback is provided according to the communication transmission module, and the voice output device and image acquisition device in the car are connected to monitor and actively question the driver, and retaliatory driving behavior is judged according to the risk score algorithm. represents a high-risk abnormal coefficient threshold, where high risk means the probability of retaliatory driving behavior is less than or equal to 90% and greater than 50%; If the abnormal coefficient is in the fourth abnormal interval If the driver's driving behavior, physiological and psychological state are abnormal and at a significant risk, and the driver is highly likely to engage in retaliatory driving behavior, the communication transmission module will provide feedback on the dangerous state and initiate active intervention operations; If it is determined that the driver is in a dangerous driving state, an active intervention operation is performed based on the vehicle driving state data, and then a dangerous driving alarm and a dangerous driving feedback report are issued based on the driver's driving state data. The active intervention operation is performed based on the vehicle driving radar data.

2. The vehicle warning method for identifying dangerous driving behavior of a driver according to claim 1, characterized in that: The multi-view anomaly detection algorithm and the risk scoring algorithm specifically include: Multi-view anomaly detection algorithms include local anomaly detection algorithms and global anomaly detection algorithms; The local anomaly detection score of all indicators in the vehicle driving state data and the driver driving state data at the current moment is calculated according to the local anomaly detection algorithm. The specific algorithm of the local anomaly detection algorithm is as follows: ， in, represents the local anomaly detection score, Represents the data points in the vehicle driving status data and the driver driving status data, express The neighbor set of express The number of neighbors of express The neighboring data points of express and The Euclidean distance of The global anomaly detection score of all indicators in the vehicle driving state data and the driver driving state data at the current moment is calculated according to the global anomaly detection algorithm. The specific algorithm of the global anomaly detection algorithm is as follows: in, represents the global anomaly detection score, Indicates indicators in vehicle driving status data and driver driving status data, Indicates the total number of indicators, express The probability density of The anomaly coefficient is calculated based on the local anomaly detection score and the global anomaly detection score. The specific algorithm for calculating the anomaly coefficient is as follows: in, represents the anomaly coefficient, Indicates normalization processing; The specific formula of the risk scoring algorithm is as follows: ， ， in, 、 、 Respectively represent the driver status index, vehicle status index, and environmental factor index, 、 、 、 Represent different indicator risk ranges, Indicates that the driver is in good condition, the vehicle is in good condition, and there are few pedestrians and obstacles in the environment. The said few pedestrians and obstacles in the environment means that there are no pedestrians and obstacles in the vehicle radar range. , Indicates that the driver's state is average, the vehicle's state is average, and there are few pedestrians and obstacles in the environment. The few pedestrians and obstacles in the environment means that there are few pedestrians and obstacles in the vehicle radar range. , Indicates that the driver's condition is poor, the vehicle's condition is poor, and there are many pedestrians and obstacles in the environment. The presence of many pedestrians and obstacles in the environment indicates that there are many pedestrians and obstacles in the vehicle's radar range. , Indicates that the driver's condition is extremely bad, the vehicle's condition is extremely bad, and there are dense pedestrians and obstacles in the environment. The dense pedestrians and obstacles in the environment indicate that there are dense pedestrians and obstacles in the vehicle radar range. The good, average, poor and extremely poor driver and vehicle states correspond to different fluctuation levels of the driver state index and vehicle state index, respectively. Represents the output score of a single risk score of the indicator data, Represents the total risk score.

3. The vehicle warning method for identifying dangerous driving behavior of a driver according to claim 1, characterized in that: The multimodal feature extraction subnetwork specifically includes: Obtaining a current driver's steering wheel pressure image based on the steering wheel pressure sensor; Obtain the driver's facial information image at the current moment according to the in-car image collector; Obtain the current driver's voice information based on the in-car audio collector; performing image registration processing on the steering wheel touch pressure image and the driver's facial information image, and then performing image synthesis, wherein the image synthesis is performed by performing wavelet decomposition on the steering wheel touch pressure image and the driver's facial information image based on wavelet transform to obtain multiple sub-images in different frequency bands, and fusing them to generate a synthesized image; performing an image feature extraction operation on the synthesized image according to a convolutional neural sub-network to obtain basic image features of the driver, and then performing feature enhancement on the basic image features of the driver according to a local attention enhancement module to obtain enhanced image features of the driver; Performing an audio feature extraction operation on the driver's voice information according to the Mel-frequency cepstral coefficient to obtain the driver's basic audio features, and then performing feature enhancement on the driver's basic audio features according to the key voice attention enhancement module to obtain the driver's enhanced audio features; The driver's text information is generated according to the conversion of the driver's voice information, and then a text feature extraction operation is performed on the driver's voice information according to the global vector word embedding to obtain the driver's basic text features, and then the driver's basic text features are enhanced according to the keyword attention enhancement module to obtain the driver's enhanced text features.

4. The vehicle warning method for identifying dangerous driving behavior of a driver according to claim 1, characterized in that: The multimodal feature fusion sub-network specifically includes: The multimodal feature fusion sub-network includes a multi-layer long short-term memory network, and the multi-layer long short-term memory network includes three network layers; Inputting the driver enhancement image features obtained by the multimodal feature extraction subnetwork into the first network layer to obtain the first hidden layer state features of each neuron in the first network layer; splicing the driver enhanced audio features obtained by the multimodal feature extraction subnetwork and the first hidden layer state features and inputting the concatenated features into the second network layer to obtain the second hidden layer state features of each neuron in the second network layer; splicing the driver enhanced text features obtained by the multimodal feature extraction subnetwork and the second hidden layer state features and inputting the concatenated features into the third network layer to obtain the third hidden layer state features; A multimodal fusion feature is obtained based on the third hidden layer state feature, and then the driver's dangerous driving behavior is judged based on the multimodal fusion feature to obtain a classification prediction result of the driver's dangerous driving behavior judgment.

5. The vehicle warning method for identifying dangerous driving behavior of a driver according to claim 1, characterized in that: If it is determined that the driver is in a dangerous driving state, the step of performing active intervention operations according to the vehicle driving state data specifically includes: Actively intervening based on vehicle driving radar data in the vehicle driving status data, wherein the active intervention is performed according to a preset active intervention strategy; The specific algorithm of the preset active intervention strategy is as follows: ， in, Indicates the preset active intervention strategy, Indicates the current distance between the vehicle and the vehicle, pedestrian or obstacle in front. Indicates the preset first radar distance threshold, Indicates the preset second radar distance threshold; When the active intervention determines that the current distance between the vehicle and the vehicle, pedestrian, or obstacle ahead is less than a preset first radar distance threshold, the vehicle deceleration device is controlled to decelerate the vehicle; When the active intervention determines that the current distance between the vehicle and the vehicle, pedestrian, or obstacle ahead is less than a preset second radar distance threshold, the vehicle emergency brake is controlled to bring the vehicle to a stop; When active intervention determines that the current distance between the vehicle and the vehicle, pedestrian, or obstacle ahead is less than a preset first radar distance threshold or a preset second radar distance threshold, each tire of the vehicle is individually controlled to maintain vehicle stability. When active intervention determines that the vehicle is in collision at the current moment, the airbag device in the vehicle is activated for active protection inside the vehicle, the engine hood is controlled to rise and the pedestrian protection airbag is actively opened for active protection outside the vehicle.

6. The vehicle warning method for identifying dangerous driving behavior of a driver according to claim 1, characterized in that: The step of issuing a dangerous driving warning and performing a dangerous driving feedback report based on the driver's driving status data specifically includes: When it is determined that the driver is in a dangerous driving state, the vehicle lighting equipment is controlled to flash the high and low beams alternately and turn on the hazard warning lights; Then control the vehicle's external loudspeaker and vehicle warning horn to broadcast danger avoidance warnings and issue continuous short horn sounds; Based on the driver's dangerous driving status information, driver's physiological status data and driver's emotional status data, a dangerous driving feedback report is generated and communicated back to the cloud alarm system, and an alarm message is automatically sent to the preset emergency target.

7. A vehicle warning system for identifying dangerous driving behavior of a driver, characterized by: include: An acquisition module is used to acquire vehicle driving state data and driver driving state data, wherein the vehicle driving state data includes vehicle driving radar data and vehicle driving controller data, and the driver driving state data includes driver physiological state data and driver emotional state data; an analysis and judgment module, configured to judge the driver's dangerous driving behavior based on the vehicle driving state data and the driver's driving state data, wherein the judgment of the driver's dangerous driving behavior is performed based on a comprehensive data analysis module and a multimodal feature recognition network, wherein the multimodal feature recognition network includes a multimodal feature extraction subnetwork and a multimodal feature fusion subnetwork, and the multimodal features include the driver's facial image features, speech audio features, and speech text features; The comprehensive data analysis module specifically includes: After collecting the vehicle driving status data and the driver driving status data in real time; Obtain anomaly coefficients based on a multi-view anomaly detection algorithm; If the abnormal coefficient is in the first abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are good, and the driver is in a low-risk stage. The probability of implementing retaliatory driving behavior is small, and the current judgment cycle ends. represents a low-risk abnormal coefficient threshold, where low risk means the probability of retaliatory driving behavior is less than or equal to 10%; If the abnormal coefficient is in the second abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and in a general risk stage, and the probability of implementing retaliatory driving behavior is general. The multimodal data of the current driver is collected and input into the multimodal feature recognition network as a basis for judgment, and a multimodal judgment is performed. represents the medium-risk abnormal coefficient threshold, where the medium risk means that the probability of retaliatory driving behavior is less than or equal to 50% and greater than 10%; If the abnormal coefficient is in the third abnormal interval , it is determined that the driver's driving behavior, physiological and psychological state are abnormal and at a high-risk stage, and the probability of implementing retaliatory driving behavior is high. The dangerous state feedback is provided according to the communication transmission module, and the voice output device and image acquisition device in the car are connected to monitor and actively question the driver, and retaliatory driving behavior is judged according to the risk score algorithm. represents a high-risk abnormal coefficient threshold, where high risk means the probability of retaliatory driving behavior is less than or equal to 90% and greater than 50%; If the abnormal coefficient is in the fourth abnormal interval If the driver's driving behavior, physiological and psychological state are abnormal and at a significant risk, and the driver is highly likely to engage in retaliatory driving behavior, the communication transmission module will provide feedback on the dangerous state and initiate active intervention operations; The early warning intervention module is used to perform active intervention operations based on the vehicle driving status data if it is determined that the driver is in a dangerous driving state, and then issue a dangerous driving warning and a dangerous driving feedback report based on the driver's driving status data. The active intervention operation is performed based on the vehicle driving radar data.

8. A storage medium, characterized in that: The storage medium stores one or more programs, which, when executed by the processor, implement the vehicle warning method for identifying dangerous driving behavior of a driver as described in any one of claims 1-6.

9. A computer device, characterized in that: The computer device comprises a memory and a processor, wherein: The memory is used to store computer programs; When the processor is used to execute the computer program stored in the memory, it implements the vehicle warning method for identifying dangerous driving behavior of the driver as described in any one of claims 1-6.

Images