Driving risk assessment method and device based on multi-modal data of human, vehicle and road environment

Abstract

The invention provides a driving risk assessment method and device based on human-vehicle-road environment multi-modal data, and the method comprises the steps: obtaining a plurality of driver risk factors of a driver of a target vehicle, and building a driving risk assessment model according to the plurality of driver risk factors; obtaining a weight coefficient corresponding to each driver risk factor in the driving risk assessment model; determining a risk value when the driver drives the target vehicle according to the plurality of driver risk factors and the weight coefficient corresponding to each driver risk factor; and determining a driving risk level when the driver drives the target vehicle according to the risk value.

Description

Technical Field

[0001] The present invention relates to the field of safe driving technology, and in particular to a driving risk assessment method based on multimodal data of a human-vehicle-road environment, a driving risk assessment device based on multimodal data of a human-vehicle-road environment, an electronic device, and a computer-readable storage medium. Background Art

[0002] As the traffic environment becomes increasingly complex, existing driving risk assessment technologies are no longer applicable. They cannot effectively describe the complex human-vehicle-road relationship and mechanism, nor can they accurately predict the human-vehicle-road state in time and space. Therefore, it is urgent to develop a more comprehensive and accurate driving risk assessment method to meet the challenges of the current and future traffic environment. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the first purpose of the present invention is to propose a driving risk assessment method based on multimodal data of human-vehicle-road environment, which can accurately assess driving risk by comprehensively considering multiple risk factors of the driver and dynamic and static factors in the road and traffic environment.

[0004] The second objective of the present invention is to provide a driving risk assessment device based on multimodal data of human-vehicle-road environment.

[0005] A third objective of the present invention is to provide an electronic device.

[0006] A fourth object of the present invention is to provide a computer-readable storage medium.

[0007] To achieve the above-mentioned purpose, a first embodiment of the present invention proposes a driving risk assessment method based on multimodal data of a human-vehicle-road environment, including:

[0008] Acquire multiple driver risk factors of the driver of the target vehicle, and establish a driving risk assessment model based on the multiple driver risk factors;

[0009] Obtain the weight coefficient corresponding to each driver risk factor in the driving risk assessment model;

[0010] Determine the risk value of the driver when driving the target vehicle according to multiple driver risk factors and the weight coefficient corresponding to each driver risk factor;

[0011] The driving risk level of the driver when driving the target vehicle is determined according to the risk value.

[0012] In addition, the driving risk assessment method based on multimodal data of human-vehicle-road environment according to the above embodiment of the present invention may also have the following additional technical features:

[0013] As an optional embodiment, the multiple driver risk factors include the driver's real-time driving status, real-time driving behavior, cognitive ability and personality characteristics.

[0014] As an optional embodiment, a driving risk assessment model is established based on multiple driver risk factors, including:

[0015] Determine the coupling relationship between multiple driver risk factors;

[0016] A driving risk assessment model is established based on each driver risk factor, the coupling relationship between multiple driver risk factors and the coupling relationship between driving risks.

[0017] As an optional embodiment, the risk value of the driver when driving the target vehicle is determined according to multiple driver risk factors and the weight coefficient corresponding to each driver risk factor, including:

[0018] D ri =β1D ri_1 +β2D ri_2 +β3D ri_3 +β4D ri_4

[0019] Among them, D ri is the risk value, β1 is the weight corresponding to the real-time driving state, D ri_1 is the real-time driving state, β2 is the weight corresponding to the real-time driving behavior, D ri_2 is the real-time driving behavior, β3 is the weight corresponding to cognitive ability, D ri_3 is cognitive ability, β4 is the weight corresponding to personality traits, D ri_4 For personality traits.

[0020] As an optional embodiment, determining the driving risk level of the driver when driving the target vehicle according to the risk value includes:

[0021] Compare the risk value with a preset risk threshold;

[0022] In response to the risk value being higher than a preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a first risk level;

[0023] In response to the risk value being lower than a preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a second risk level; wherein the first risk level is higher than the second risk level.

[0024] As an optional embodiment, the method further includes:

[0025] Determine the kinetic energy field generated by non-static objects around the target vehicle; wherein, non-static objects are moving objects that can actually collide with the target vehicle and cause significant damage. The kinetic energy field represents the potential degree of danger that non-static objects may cause to the surrounding environment under certain road conditions.

[0026] As an optional embodiment, the method further includes:

[0027] Determine the potential energy field generated by static objects around the target vehicle; wherein, static objects are stationary objects that can actually collide with the target vehicle and cause significant damage. The potential energy field represents the potential degree of danger that static objects may cause to the surrounding environment under certain road conditions.

[0028] As an optional embodiment, the method further includes:

[0029] The potential danger level of the driver to the surrounding environment when driving the target vehicle under certain road conditions is determined based on the kinetic energy field, the potential energy field and the risk value of the driver when driving the target vehicle.

[0030] According to the driving risk assessment method based on multimodal data of human-vehicle-road environment of an embodiment of the present invention, firstly, a plurality of driver risk factors of the driver of the target vehicle are obtained, and a driving risk assessment model is established according to the plurality of driver risk factors; further, a weight coefficient corresponding to each driver risk factor in the driving risk assessment model is obtained; further, the risk value of the driver when driving the target vehicle is determined according to the plurality of driver risk factors and the weight coefficient corresponding to each driver risk factor; finally, the driving risk level of the driver when driving the target vehicle is determined according to the risk value. The present invention realizes accurate assessment of driving risk by comprehensively considering the plurality of driver risk factors and the dynamic and static factors in the road and traffic environment.

[0031] To achieve the above-mentioned purpose, a second embodiment of the present invention proposes a driving risk assessment device based on multimodal data of a human-vehicle-road environment, the device comprising:

[0032] A first acquisition module is configured to acquire multiple driver risk factors of a target vehicle driver and establish a driving risk assessment model according to the multiple driver risk factors;

[0033] The second acquisition module is configured to obtain a weight coefficient corresponding to each driver risk factor in the driving risk assessment model;

[0034] A weight calculation module is configured to determine a risk value of a driver when driving a target vehicle according to a plurality of driver risk factors and a weight coefficient corresponding to each driver risk factor;

[0035] The evaluation module is configured to determine the driving risk level of the driver when driving the target vehicle according to the risk value.

[0036] According to a driving risk assessment device based on multimodal data of a human-vehicle-road environment provided by an embodiment of the present invention, firstly, a plurality of driver risk factors of the driver of a target vehicle are obtained, and a driving risk assessment model is established according to the plurality of driver risk factors; further, a weight coefficient corresponding to each driver risk factor in the driving risk assessment model is obtained; further, the risk value of the driver when driving the target vehicle is determined according to the plurality of driver risk factors and the weight coefficient corresponding to each driver risk factor; finally, the driving risk level of the driver when driving the target vehicle is determined according to the risk value. The present invention realizes an accurate assessment of driving risk by comprehensively considering the plurality of driver risk factors and the dynamic and static factors in the road and traffic environment.

[0037] To achieve the above-mentioned purpose, the third aspect of the present invention proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the driving risk assessment method as described above when executing the program.

[0038] According to an electronic device provided by an embodiment of the present invention, firstly, a plurality of driver risk factors of the driver of the target vehicle are obtained, and a driving risk assessment model is established according to the plurality of driver risk factors; further, a weight coefficient corresponding to each driver risk factor in the driving risk assessment model is obtained; further, the risk value of the driver when driving the target vehicle is determined according to the plurality of driver risk factors and the weight coefficient corresponding to each driver risk factor; finally, the driving risk level of the driver when driving the target vehicle is determined according to the risk value. The present invention realizes an accurate assessment of driving risk by comprehensively considering the plurality of driver risk factors and the dynamic and static factors in the road and traffic environment.

[0039] In order to achieve the above-mentioned purpose, a computer-readable storage medium is proposed in an embodiment of the fourth aspect of the present invention, and the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the above-mentioned driving risk assessment method based on multimodal data of human-vehicle-road environment.

[0040] According to a computer-readable storage medium provided by an embodiment of the present invention, firstly, a plurality of driver risk factors of the driver of the target vehicle are obtained, and a driving risk assessment model is established according to the plurality of driver risk factors; further, a weight coefficient corresponding to each driver risk factor in the driving risk assessment model is obtained; further, the risk value of the driver when driving the target vehicle is determined according to the plurality of driver risk factors and the weight coefficient corresponding to each driver risk factor; finally, the driving risk level of the driver when driving the target vehicle is determined according to the risk value. The present invention realizes an accurate assessment of driving risk by comprehensively considering the plurality of driver risk factors and the dynamic and static factors in the road and traffic environment.

[0041] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A schematic flow chart of a driving risk assessment method based on multimodal data of a human-vehicle-road environment provided in an embodiment of the present invention.

[0044] Figure 2 A schematic diagram of a driving risk assessment device based on multimodal data of a human-vehicle-road environment provided in an embodiment of the present invention.

[0045] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0047] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0048] As described in the background technology section, current driving risk assessment technologies often ignore the important impact of the driver's personality, psychological and physiological characteristics on driving safety when assessing driving safety. At the same time, they also fail to fully consider factors such as complex and changeable road conditions and traffic driving environments, and have significant limitations. Therefore, it is difficult for relevant technologies to accurately reflect the interaction and dynamic changes between driver behavior characteristics, traffic environment and vehicle status. In addition, the description of the coupling mechanism between people, vehicles and roads in relevant technologies is not clear enough. With the increasing complexity of the traffic environment, the existing safety model has become difficult to apply, and it is impossible to effectively describe the complex human-vehicle-road relationship and action mechanism, nor can it accurately predict the human-vehicle-road status in time and space changes. Therefore, it is necessary to develop a more comprehensive and accurate driving risk assessment method to meet the challenges of the current and future traffic environment.

[0049] The technical solution of the present invention is further described in detail below through specific embodiments.

[0050] refer to Figure 1 , which is a flow chart of a driving risk assessment method based on multimodal data of a human-vehicle-road environment provided in an embodiment of the present invention.

[0051] Step S101, obtaining multiple driver risk factors of the driver of the target vehicle, and establishing a driving risk assessment model according to the multiple driver risk factors;

[0052] As an optional embodiment, the multiple driver risk factors include the driver's real-time driving status, real-time driving behavior, cognitive ability and personality characteristics.

[0053] In many cases, drivers are the main factor leading to traffic accidents. As key participants in the traffic system, their behavior, decision-making and reaction speed are directly related to the safety of road traffic.

[0054] In specific implementations, the driver's real-time driving status may be affected by a variety of factors. From the driver's own conditions, it may include physical fatigue, excitement, pain, etc., and psychological anger, anxiety, tension, etc. For example, long-term continuous driving or lack of sleep can cause driver fatigue, affect their judgment and reaction speed, and thus increase the risk of accidents. When the driver is in a negative emotional state such as anger, anxiety, and tension, it is easy to make wrong driving decisions, such as sudden braking, slamming the steering wheel, etc., which may cause accidents. Suffering from diseases such as heart disease and high blood pressure, or sudden illness during driving, may affect the driver's driving ability and reaction speed.

[0055] To obtain the driver's real-time driving status, the driver's physiological data, including but not limited to blinking frequency, gaze direction, facial expression changes, heart rate, etc., can be collected in real time through biometric sensors installed in the vehicle (such as eye trackers, facial expression recognition cameras, heart rate monitors, etc.). Among them, the eye tracker can determine the driver's gaze direction, concentration, and fatigue level by tracking the movement trajectory of the eyeballs. When the driver's eyes deviate from the road or do not blink for a long time, it can be determined that the driver is in a state of fatigue. The facial expression recognition camera can identify the driver's emotional state, such as tension, relaxation, fatigue, etc., by analyzing changes in facial expressions. The heart rate monitor can assess the driver's physiological stress level and health status by monitoring changes in heart rate. When the heart rate rises abnormally, the system can determine that the driver may be in a state of tension or anxiety.

[0056] It should be noted that the driver's psychological emotions can often be reflected through the above-mentioned physiological characteristics. Therefore, the driver's psychological state can be mapped out through the captured physiological data.

[0057] The driver's real-time driving behavior is also one of the factors that affect driving safety, such as speeding, illegal operation, distracted driving, etc. Specifically, speeding is one of the main causes of traffic accidents. When the speed is too fast, the driver's reaction time will become shorter, increasing the risk of collision. Failure to comply with traffic rules, such as running red lights, driving in the wrong direction, and changing lanes at will, will increase the risk of collision with other vehicles or pedestrians. Distracting behaviors such as using mobile phones, eating, and talking to passengers will distract the driver's attention and make it impossible to respond to emergencies in time.

[0058] To determine the driver's real-time driving behavior, existing vehicle sensors (such as accelerometers, gyroscopes, pressure sensors, etc.) can be installed or used to record the strength, stability, and accuracy of acceleration, deceleration, and steering operations. In-car cameras or external cameras (under the premise of privacy compliance) can also be used to capture the driver's posture, facial expressions, etc. to analyze the standardization of driving posture. Vehicle operating status data, including speed, engine status, etc., can also be obtained through the vehicle's diagnostic interface.

[0059] The driver's cognitive ability can characterize the driver's ability to accurately perceive traffic information and make reasonable judgments and decisions. For example, when a driver is driving on the road, the choice and execution of overtaking methods require good cognitive ability. For example, when a driver decides to overtake, he must accurately estimate his own speed, the speed of the vehicle being overtaken, and the speed of other related vehicles on the road. At the same time, the driver also needs to control the vehicle when overtaking in accordance with the dynamic characteristics of the car itself, such as choosing the appropriate gear. If the driver's judgment in these aspects is inaccurate, it may cause a traffic accident. This reflects the driver's cognitive ability of speed, distance and vehicle control during overtaking. When observing the road line shape, the driver is prone to illusion of curves and slopes, which requires the driver to have good spatial cognitive ability and visual perception ability. For example, in a continuous curve, because the number of direction changes is large, the driver may feel that the curve is more curved than the actual curvature, which is the illusion of curves. If the driver cannot accurately judge the road line shape, it may cause the vehicle to lose control or a traffic accident. Similarly, if the slope changes on a long downhill road, it is easy to give people an illusion. The driver needs to accurately judge the slope change to adjust the speed and keep the vehicle stable. The driver needs to pay attention to the vehicle blind spot and the environmental blind spot at all times during driving. Not knowing the existence of vehicle blind spots and environmental blind spots usually occurs in novices. This type of novice is fearless when driving, with only himself in mind, no one else, running around, changing lanes, turning, turning around, braking, and slowing down at will, which is very dangerous. Some experienced drivers, although they know the existence of blind spots, may not know that there may be cars, people, obstacles, etc. in the blind spots, which may also lead to traffic accidents. Therefore, drivers need to continuously improve their cognitive ability of blind spots, remind other road users by honking horns, flashing lights, etc., and pay attention to the surrounding environment to ensure driving safety. When driving fatigued, the driver's perception, reaction and judgment abilities will be significantly reduced. For example, the driver cannot make accurate judgments on spatial distance, the distance of obstacles and the speed of the vehicle, which may cause the vehicle to lose control or cause a traffic accident. In addition, fatigue driving may also lead to driver errors, such as untimely and inaccurate gear shifting, slow operation, etc., which may endanger driving safety. Therefore, drivers need to arrange driving time and rest time reasonably to avoid fatigue driving. A mature driver will find an optimal balance between traffic laws and common sense, that is, he will abide by traffic rules himself but not make ideal requirements for others, and incorporate other people's violations and unexpected behaviors on the road into his regular driving response operations. For example, at a railway crossing, the driver should stop, look, listen and pass, which is a manifestation of compliance with traffic rules. At the same time, when encountering emergencies during driving, the driver needs to make judgments quickly and take corresponding response measures, such as emergency braking and avoidance. This requires the driver to have good reaction and judgment abilities.

[0060] The cognitive ability of the driver can be determined through observation or indirect evaluation. For example, the driver's driving behavior can be observed. During the actual driving process, the driver's acceleration, deceleration, steering and other operating behaviors can be observed to evaluate their strength, stability and accuracy, and the driver's reaction speed and decision-making ability when dealing with emergencies can be noted. Driving posture and habits can also be observed, such as observing the driver's seat adjustment, the position of the hand holding the steering wheel, etc., to evaluate the standardization of his driving posture, and analyze the driver's driving habits, such as whether he changes lanes frequently, whether he obeys traffic rules, etc. For another example, psychological tests and evaluations can be conducted on the driver in advance, such as cognitive ability tests, that is, graphic recognition, orientation recognition, color recognition and other tests, to evaluate the driver's spatial perception, visual memory and color recognition ability. The driver's mental agility and logical judgment can also be evaluated through digital calculation or logical reasoning questions. Attention and reaction tests can also be conducted, that is, using professional testing tools or software to measure the driver's concentration and reaction speed. The driver's reaction and response strategy can be observed by simulating emergency or sudden situations.

[0061] In addition, the driver's personality traits are also one of the factors in driving risk assessment. Drivers with higher impulsiveness tend to be more inclined to take risks, such as speeding, running red lights or not obeying traffic rules. These behaviors significantly increase the possibility of traffic accidents. Drivers with greater emotional fluctuations may overreact when faced with stress or emergencies, such as sudden acceleration, sudden braking or emotional loss of control. These behaviors can easily lead to traffic accidents. Drivers with a strong sense of responsibility usually pay more attention to driving safety. They not only obey traffic rules, but also regularly check the condition of the vehicle to ensure that the vehicle is in good driving condition. On the contrary, drivers who lack a sense of responsibility may be more likely to ignore safety issues, thereby increasing the risk of accidents.

[0062] Similarly, the personality traits of drivers can also be comprehensively evaluated by observing driving behavior, evaluating driving attitudes, analyzing the way drivers interact with others, or using psychological tests and assessment tools. For example, the attitude and behavior of drivers when interacting with other drivers can be observed through relevant detection sensors. Or the communication and interaction between drivers and passengers can be analyzed. Whether the driver is enthusiastic and helpful may reflect the enthusiasm and sociability of his personality.

[0063] The present invention can timely detect potential driving risks and issue warnings to drivers by real-time monitoring of multiple risk factors of drivers and the surrounding environment of the vehicle, thereby effectively avoiding or reducing the occurrence of traffic accidents. According to individual differences of drivers (such as cognitive ability, personality characteristics, etc.), more personalized risk assessment results can be provided, so that drivers can better understand their own driving risks and take more effective preventive measures.

[0064] It should be noted that before using the technical solutions disclosed in the embodiments of the present invention, the type, scope of use, usage scenarios, etc. of the personal information involved in the present invention should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0065] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present invention according to the prompt message.

[0066] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0067] It is understandable that the above notification and user authorization process is only illustrative and does not constitute a limitation on the implementation of the present invention. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present invention.

[0068] Step S102, obtaining the weight coefficient corresponding to each driver risk factor in the driving risk assessment model.

[0069] As an optional embodiment, a driving risk assessment model is established based on multiple driver risk factors, including: determining the coupling relationship between the multiple driver risk factors; establishing a driving risk assessment model based on each driver risk factor, the coupling relationship between the multiple driver risk factors and the coupling relationship between driving risks.

[0070] In specific implementation, the real-time driving status, real-time driving behavior, cognitive ability and personality characteristics of the driver collected from the above multiple aspects can be integrated and processed, and modeled and analyzed using intelligent algorithms (such as convolutional neural network CNN and recurrent neural network RNN ​​in deep learning algorithms). Through learning and training of actual driving data, the inherent correlation between different states of the driver and the complex relationship between them and driving risks are excavated, so as to establish a risk assessment model with high accuracy and adaptability, and quantify the weight corresponding to each driver risk factor in the driving risk assessment model.

[0071] Specifically, the raw data (i.e., real-time driving status, real-time driving behavior, cognitive ability, and personality characteristics) collected from multiple sensors and test tools are cleaned, denoised, and formatted to ensure the accuracy and consistency of the data. Interpolation processing is performed on missing data, and abnormal data is filtered or corrected. Features that have a significant impact on driving risks are extracted from the preprocessed data, such as changes in driving speed, steering wheel rotation angle, accelerator pedal pedaling force, driver's concentration, reaction speed, emotional stability index, etc. At the same time, personality characteristic data (such as personality type and risk preference obtained through psychological tests) are also extracted as one of the features. The extracted feature data are fused to form a comprehensive data set containing multi-dimensional information. Data splicing, feature mapping, and other methods can be used to integrate data from different sources into a unified framework. Further, according to the complexity of the problem and the characteristics of the data, a suitable intelligent algorithm is selected for modeling and analysis. In an embodiment of the present invention, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) in deep learning algorithms can be used. Further, a deep learning model is constructed using the extracted feature data. For CNN, it can be used to process image data (such as facial expressions and gestures of drivers obtained through driving recorders) to extract visual features of drivers. For RNN, it can be used to process time series data (such as driving speed, steering wheel rotation angle and other data that change over time) to capture the dynamic behavior characteristics of drivers. Furthermore, the model is trained using actual driving data so that it can learn the intrinsic associations between different states of drivers and the complex relationship patterns between them and driving risks. During the training process, the parameters and structure of the model need to be continuously adjusted to improve its prediction performance. The model is verified by cross-validation, test set evaluation and other methods to ensure the accuracy and reliability of its prediction results. The model is optimized and adjusted according to the verification results, such as increasing the number of network layers and adjusting the learning rate.

[0072] Step S103, determining the risk value of the driver when driving the target vehicle according to the multiple driver risk factors and the weight coefficient corresponding to each driver risk factor.

[0073] As an optional embodiment, the risk value of the driver when driving the target vehicle is determined according to multiple driver risk factors and the weight coefficient corresponding to each driver risk factor, including:

[0074] D ri =β1D ri_1 +β2D ri_2 +β3D ri_3 +β4D ri_4

[0075] Among them, D riis the risk value, β1 is the weight corresponding to the real-time driving state, D ri_1 is the real-time driving state, β2 is the weight corresponding to the real-time driving behavior, D ri_2 is the real-time driving behavior, β3 is the weight corresponding to cognitive ability, D ri_3 is cognitive ability, β4 is the weight corresponding to personality traits, D ri_4 For personality traits.

[0076] Among them, β1+β2+β3+β4=1.

[0077] According to the calculated D ri The driving risk of the driver is scored and graded. ri =0.25, indicating a low risk level, with a score of 25.

[0078] Among them, each driver risk factor may include several levels, and each level may correspond to its own sub-weight. When a certain driver risk factor includes multiple levels, the weight corresponding to the driver risk factor will be the accumulation of the sub-weights corresponding to the multiple levels. For example, for the real-time driving state, when the real-time driving state includes both fatigue and distraction, the weight coefficient corresponding to the real-time driving state will be the accumulation of the sub-weight corresponding to fatigue and the sub-weight corresponding to distraction, that is, the weight coefficient corresponding to the real-time driving state will increase when calculating the risk value of the driver when driving the target vehicle. By grading each driver risk factor and assigning sub-weights to each level, the fine-grained recognition accuracy in the case of less data can be increased, and the risk value of the driver when driving the target vehicle can be determined more accurately based on multiple driver risk factors and the weight coefficients corresponding to each driver risk factor.

[0079] Step S104, determining the driving risk level of the driver when driving the target vehicle according to the risk value.

[0080] As an optional embodiment, determining the driving risk level of the driver when driving the target vehicle based on the risk value includes: comparing the risk value with a preset risk threshold; in response to the risk value being higher than the preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a first risk level; in response to the risk value being lower than the preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a second risk level; wherein the first risk level is higher than the second risk level.

[0081] Specifically, one or more preset risk thresholds may be set first. These thresholds may be determined based on factors such as historical data, industry standards, regulatory requirements or expert opinions, and are intended to divide the risk value into different intervals to correspond to different risk levels. Next, the calculated risk value is compared with the preset risk threshold. This comparison process is to determine in which risk interval the risk value falls, thereby further determining the driving risk level. If the risk value is higher than the preset risk threshold, it means that the driver's driving risk under the current conditions is high, so the driving risk level is determined to be the first risk level. This level usually indicates that special attention or emergency measures are needed to reduce the risk. If the risk value is lower than the preset risk threshold, it means that the driver's driving risk under the current conditions is relatively low, so the driving risk level is determined to be the second risk level. This level usually indicates that the driver's driving behavior is relatively safe, but still needs to remain vigilant. In an embodiment of the present invention, the first risk level is set to be higher than the second risk level, which means that when the risk value is high, the driver's driving risk is also higher, and more attention and corresponding safety measures need to be taken.

[0082] It should be noted that the preset risk threshold is not static, but can be adjusted dynamically according to actual conditions. For example, when new regulatory requirements or industry standards are introduced, the preset risk threshold can be adjusted accordingly according to these requirements or standards. When determining the driving risk level, in addition to considering the risk value, other factors can also be considered comprehensively, such as the driver's historical driving record, vehicle condition, road conditions, etc. These factors can provide more comprehensive risk assessment information and help determine the driving risk level more accurately.

[0083] As an optional embodiment, the kinetic energy field generated by non-static objects around the target vehicle can also be determined; wherein the non-static objects are moving objects that can actually collide with the target vehicle and cause greater losses, and the kinetic energy field represents the potential degree of danger that the non-static objects pose to the surrounding environment under certain road conditions.

[0084] Specifically, the kinetic field represents moving objects that can actually collide with the vehicle and cause greater losses, such as moving vehicles and pedestrians. The kinetic risk field of all moving objects in the driving scene is calculated using the following formula based on the real-time data collected from the vehicle and other vehicles, pedestrians, and roads. The danger brought by the collision does not increase linearly, but as the distance between the two decreases, the speed at which the driving danger increases accelerates. Therefore, the exponential form of this increase in collision risk is used. Based on the above analysis, the non-static object i(x i ,y i ) forms a kinetic energy field around it at (x j ,y j ) is:

[0085]

[0086] Among them, x j ,y j is the coordinate of the center of mass of the non-static object, the x-axis is along the road line direction, and the y-axis is perpendicular to the road line direction; vector E V_ij It represents the potential danger of non-static object i to the surrounding environment under certain road conditions. The greater the field strength, the greater the potential danger of non-static object i. The direction of the field strength is related to r ij The same, and along the direction of field strength, the field strength decreases fastest; r ij =(x j -x i ,y j -y i ) represents the distance vector between two points; k1, k2, G are all unknown constants greater than 0; M i is the equivalent mass of object i; R i is (x i ,y i ) is the factor affecting the road conditions; object i moves along the positive x-axis, v i is the velocity of object i; θ i is the velocity direction of object i and r i The angle of j is positive in the clockwise direction.

[0087] Based on the above analysis, the kinetic energy field of the vehicle and the risk value D of the above driver risk factor are combined. ri Do the product and you get the non-static object (vehicle) i(x i ,y i )'s driver forms a behavioral field around him in (x j ,y j ) Field strength:

[0088] E D_ij =E V_ij D ri

[0089] Among them, E V_ij The kinetic energy field formed by vehicle i is (x j ,y j ) at the point where the field strength is; D ri is the risk value D of the driver's risk factor ri .

[0090] Vector D_ij It represents the potential danger degree caused by the driver of vehicle i to the surrounding environment when driving a certain vehicle under certain road conditions. The greater the field strength, the greater the potential danger caused by the driver. The direction of the field strength is the same as the direction of the kinetic energy field formed by vehicle i.

[0091] As an optional embodiment, the potential energy field generated by static objects around the target vehicle can also be determined; wherein the static objects are stationary objects that can actually collide with the target vehicle and cause significant damage, and the potential energy field represents the potential degree of danger that the static objects pose to the surrounding environment under certain road conditions.

[0092] Static object i(x i ,y i ) forms a potential energy field around it at (x j ,y j ) is:

[0093]

[0094] Among them, x j ,y j is the coordinate of the center of mass of the static object, the x-axis is along the road line direction, and the y-axis is perpendicular to the road line direction; vector E R_ij It represents the potential danger degree of static object i to the surrounding environment under certain road conditions. The greater the field strength, the greater the potential danger caused by static object i. The direction of the field strength is related to r ij The same, and along the direction of field strength, the field strength decreases fastest; r ij =(x j -x i ,y j -y i ) represents the distance vector between two points; k1, k2, G are all unknown constants greater than 0; M i is the equivalent mass of object i; R i is (x i ,y i ) is the influencing factor of road conditions.

[0095] As an optional embodiment, the potential danger level of the driver to the surrounding environment when driving the target vehicle under certain road conditions can be determined based on the kinetic energy field, the potential energy field and the risk value of the driver when driving the target vehicle.

[0096] In summary, the unified model of driving risk field can be expressed as:

[0097]

[0098] Among them, F j Indicates the degree of risk of vehicle j’s current driving state, F j The larger the value, the more dangerous the current driving state is. Its direction is similar to E j same.

[0099] From the above, it can be seen that the driving risk assessment method based on multimodal data of human-vehicle-road environment provided by the present invention first obtains multiple driver risk factors of the driver of the target vehicle, and establishes a driving risk assessment model based on the multiple driver risk factors; further, obtains the weight coefficient corresponding to each driver risk factor in the driving risk assessment model; further, determines the risk value of the driver when driving the target vehicle based on multiple driver risk factors and the weight coefficient corresponding to each driver risk factor; finally, determines the driving risk level of the driver when driving the target vehicle based on the risk value. The present invention realizes accurate assessment of driving risks by comprehensively considering multiple risk factors of the driver and dynamic and static factors in the road and traffic environment.

[0100] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.

[0101] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0102] Based on the same inventive concept, corresponding to the method provided in any of the above embodiments, the present invention also provides a driving risk assessment device based on multimodal data of a human-vehicle-road environment.

[0103] refer to Figure 2 , which is a schematic diagram of a driving risk assessment device based on multimodal data of a human-vehicle-road environment provided in an embodiment of the present invention.

[0104] The device comprises:

[0105] The first acquisition module 201 is configured to acquire multiple driver risk factors of the target vehicle driver and establish a driving risk assessment model according to the multiple driver risk factors;

[0106] The second acquisition module 202 is configured to obtain a weight coefficient corresponding to each driver risk factor in the driving risk assessment model;

[0107] A weight calculation module 203 is configured to determine the risk value of the driver when driving the target vehicle according to a plurality of driver risk factors and a weight coefficient corresponding to each driver risk factor;

[0108] The evaluation module 204 is configured to determine the driving risk level of the driver when driving the target vehicle according to the risk value.

[0109] Optionally, the multiple driver risk factors include the driver's real-time driving status, real-time driving behavior, cognitive ability and personality characteristics.

[0110] Optionally, the first acquisition module 201 is further configured to:

[0111] Determine the coupling relationship between multiple driver risk factors;

[0112] A driving risk assessment model is established based on each driver risk factor, the coupling relationship between multiple driver risk factors and the coupling relationship between driving risks.

[0113] Optionally, the weight calculation module 203 is further configured to:

[0114] D ri =β1D ri_1 +β2D ri_2 +β3D ri_3 +β4D ri_4

[0115] Among them, D ri is the risk value, β1 is the weight corresponding to the real-time driving state, D ri_1 is the real-time driving state, β2 is the weight corresponding to the real-time driving behavior, D ri_2 is the real-time driving behavior, β3 is the weight corresponding to cognitive ability, D ri_3 is cognitive ability, β4 is the weight corresponding to personality traits, D ri_4 For personality traits.

[0116] Optionally, the evaluation module 204 is further configured to:

[0117] Compare the risk value with a preset risk threshold;

[0118] In response to the risk value being higher than a preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a first risk level;

[0119] In response to the risk value being lower than a preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a second risk level; wherein the first risk level is higher than the second risk level.

[0120] Optionally, the evaluation module 204 is further configured to:

[0121] Determine the kinetic energy field generated by non-static objects around the target vehicle; wherein, non-static objects are moving objects that can actually collide with the target vehicle and cause significant damage. The kinetic energy field represents the potential degree of danger that non-static objects may cause to the surrounding environment under certain road conditions.

[0122] Optionally, the evaluation module 204 is further configured to:

[0123] Determine the potential energy field generated by static objects around the target vehicle; wherein, static objects are stationary objects that can actually collide with the target vehicle and cause significant damage. The potential energy field represents the potential degree of danger that static objects may cause to the surrounding environment under certain road conditions.

[0124] Optionally, the evaluation module 204 is further configured to:

[0125] The potential danger level of the driver to the surrounding environment when driving the target vehicle under certain road conditions is determined based on the kinetic energy field, the potential energy field and the risk value of the driver when driving the target vehicle.

[0126] According to a driving risk assessment device based on multimodal data of a human-vehicle-road environment provided by an embodiment of the present invention, firstly, a plurality of driver risk factors of the driver of a target vehicle are obtained, and a driving risk assessment model is established according to the plurality of driver risk factors; further, a weight coefficient corresponding to each driver risk factor in the driving risk assessment model is obtained; further, the risk value of the driver when driving the target vehicle is determined according to the plurality of driver risk factors and the weight coefficient corresponding to each driver risk factor; finally, the driving risk level of the driver when driving the target vehicle is determined according to the risk value. The present invention realizes an accurate assessment of driving risk by comprehensively considering the plurality of driver risk factors and the dynamic and static factors in the road and traffic environment.

[0127] For the convenience of description, the above system is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0128] The system of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0129] Based on the same inventive concept, corresponding to the method described in any of the above embodiments, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the program.

[0130] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 310, a memory 320, an input / output interface 330, a communication interface 340, and a bus 350. The processor 310, the memory 320, the input / output interface 330, and the communication interface 340 are connected to each other in communication within the device through the bus 350.

[0131] The processor 310 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0132] The memory 320 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 320 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 320 and are called and executed by the processor 310.

[0133] The input / output interface 330 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0134] The communication interface 340 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0135] The bus 350 comprises a pathway that transmits information between the various components of the device (eg, the processor 310 , the memory 320 , the input / output interface 330 , and the communication interface 340 ).

[0136] It should be noted that, although the above device only shows the processor 310, the memory 320, the input / output interface 330, the communication interface 340 and the bus 350, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0137] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0138] Based on the same inventive concept, corresponding to the method described in any of the above embodiments, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0139] The above-mentioned computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0140] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method described in any embodiment of the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0141] In addition, although the operations of the method of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0142] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0143] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the embodiments of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0144] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and this division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. A driving risk assessment method based on multimodal data of human-vehicle-road environment, characterized in that: include: Acquire multiple driver risk factors of the driver of the target vehicle, and establish a driving risk assessment model according to the multiple driver risk factors; Obtaining a weight coefficient corresponding to each driver risk factor in the driving risk assessment model; Determining the risk value of the driver when driving the target vehicle according to a plurality of driver risk factors and a weight coefficient corresponding to each driver risk factor; The driving risk level of the driver when driving the target vehicle is determined according to the risk value.

2. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 1 is characterized in that: The multiple driver risk factors include the driver's real-time driving status, real-time driving behavior, cognitive ability and personality characteristics.

3. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 2 is characterized in that: The establishing of a driving risk assessment model according to the multiple driver risk factors includes: Determining a coupling relationship between the plurality of driver risk factors; The driving risk assessment model is established according to each of the driver risk factors, the coupling relationship between the multiple driver risk factors and the coupling relationship between driving risks.

4. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 3 is characterized in that: Determining the risk value of the driver when driving the target vehicle according to the multiple driver risk factors and the weight coefficient corresponding to each driver risk factor includes: D ri =β1D ri_1 +β2D ri_2 +β3D ri_3 +β4D ri_4 Among them, D ri is the risk value, β1 is the weight corresponding to the real-time driving state, D ri_1 is the real-time driving state, β2 is the weight corresponding to the real-time driving behavior, D ri_2 is the real-time driving behavior, β3 is the weight corresponding to cognitive ability, D ri_3 is cognitive ability, β4 is the weight corresponding to personality traits, D ri_4 For personality traits.

5. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 4 is characterized in that: The determining, according to the risk value, the driving risk level of the driver when driving the target vehicle includes: comparing the risk value with a preset risk threshold; In response to the risk value being higher than a preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a first risk level; In response to the risk value being lower than a preset risk threshold, determining that the driving risk level of the driver when driving the target vehicle is a second risk level; wherein the first risk level is higher than the second risk level.

6. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 1 is characterized in that: The method further comprises: Determine the kinetic energy field generated by non-static objects around the target vehicle; wherein the non-static objects are moving objects that can actually collide with the target vehicle and cause significant damage, and the kinetic energy field represents the potential degree of danger that the non-static objects may cause to the surrounding environment under certain road conditions.

7. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 6 is characterized in that: The method further comprises: Determine the potential energy field generated by static objects around the target vehicle; wherein the static objects are stationary objects that can actually collide with the target vehicle and cause significant damage, and the potential energy field represents the potential degree of danger that the static objects may cause to the surrounding environment under certain road conditions.

8. The driving risk assessment method based on multimodal data of human-vehicle-road environment according to claim 7 is characterized in that: The method further comprises: The potential danger level of the driver to the surrounding environment when driving the target vehicle under certain road conditions is determined according to the kinetic energy field, the potential energy field and the risk value of the driver when driving the target vehicle.

9. A driving risk assessment device based on multimodal data of human-vehicle-road environment, characterized in that: include: A first acquisition module is configured to acquire multiple driver risk factors of a target vehicle driver and establish a driving risk assessment model according to the multiple driver risk factors; A second acquisition module is configured to obtain a weight coefficient corresponding to each driver risk factor in the driving risk assessment model; A weight calculation module is configured to determine the risk value of the driver when driving the target vehicle according to the multiple driver risk factors and the weight coefficient corresponding to each of the driver risk factors; The evaluation module is configured to determine the driving risk level of the driver when driving the target vehicle according to the risk value.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the driving risk assessment method based on multimodal data of human-vehicle-road environment as described in any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to implement the driving risk assessment method based on multimodal data of human-vehicle-road environment as described in any one of claims 1 to 8.

Images