Driving risk dynamic early warning method and system based on human factor and road condition fusion

Abstract

The application provides a driving risk dynamic early warning method and system based on human factors and road conditions, and relates to the technical field of driving safety. Firstly, dynamic state data of drivers and security guards and real-time state data of roads during driving are collected to generate a human factor state dynamic evolution sequence and a road condition real-time change sequence. An interaction influence relationship between the two is analyzed to establish a human factor and road condition bidirectional correlation model. A risk trigger condition is extracted based on the human factor and road condition bidirectional correlation model to identify a driving risk conduction path set. Feature extraction is performed on the driving risk conduction path set to determine a risk influence range, thereby generating a driving risk dynamic early warning instruction and sending the driving risk dynamic early warning instruction to a driving control terminal to trigger an early warning prompt, so that driving risks can be effectively and accurately early warned, and driving safety is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of driving safety, in particular to a driving risk dynamic early warning method and system based on the fusion of human factors and road conditions. BACKGROUND

[0002] In the field of driving safety, it is crucial to ensure the safety during vehicle driving. At present, the existing driving risk early warning methods mostly only focus on the monitoring and analysis of road conditions, such as obtaining the traffic flow, road flatness and other information of the road through sensors to determine whether there is a potential risk. However, the above early warning method which simply depends on the road conditions ignores the important influence of human factors such as the driver and the escort on driving safety. The fatigue degree, attention concentration, emotional state of the driver and the cooperation state of the escort may cause different driving risks under different road conditions, but the existing technology fails to include the above key human factor information in the early warning consideration range.

[0003] On the other hand, some methods consider human factors, but often only analyze the state of the driver or the escort in isolation, without organically combining human factors and road conditions. Human factor state and road conditions are interrelated and influence each other. Different human factor states will produce different risk results under different road conditions, and the existing technology lacks effective analysis and modeling of the above complex interaction, which leads to the inability to accurately identify the transmission path and influence range of driving risk, and thus it is difficult to provide accurate and effective driving risk early warning, which brings potential hidden dangers to driving safety. SUMMARY

[0004] In view of the above-mentioned problems, in combination with the first aspect of the present application, the embodiments of the present application provide a driving risk dynamic early warning method based on the fusion of human factors and road conditions, which comprises:

[0005] Collecting dynamic state data of the driver, dynamic state data of the escort and real-time state data of the road during driving, generating a human factor state dynamic evolution sequence based on the collected dynamic state data of the driver and the dynamic state data of the escort, and generating a road condition real-time change sequence based on the collected real-time state data of the road;

[0006] Analyzing the interaction influence relationship between each state node in the human factor state dynamic evolution sequence and each road condition node in the road condition real-time change sequence, and establishing a human factor-road condition bidirectional correlation model;

[0007] Based on the human factor-road condition bidirectional correlation model, extracting the risk trigger conditions under different state node and road condition node combinations, and identifying a driving risk transmission path set;

[0008] extract path features of each driving risk conduction path, and determine a risk influence range corresponding to each driving risk conduction path according to the extracted path features;

[0009] In combination with the risk influence range corresponding to each driving risk conduction path, a driving risk dynamic early warning instruction containing a risk conduction direction and a risk influence range description is generated, and the driving risk dynamic early warning instruction is sent to a driving control terminal to trigger the driving control terminal to perform a warning prompt operation.

[0010] In still another aspect, the embodiment of the present application further provides a driving risk dynamic early warning system based on fusion of human factors and road conditions, characterized in that it comprises:

[0011] a processor; a machine readable storage medium for storing machine executable instructions of the processor; wherein the processor is configured to execute the machine executable instructions to perform the above-mentioned driving risk dynamic early warning method based on fusion of human factors and road conditions.

[0012] In still another aspect, the embodiment of the present application further provides a computer program product, which comprises machine executable instructions stored in a computer readable storage medium, and a processor of a driving risk dynamic early warning system based on fusion of human factors and road conditions reads the machine executable instructions from the computer readable storage medium, and the processor executes the machine executable instructions to enable the driving risk dynamic early warning system based on fusion of human factors and road conditions to perform the above-mentioned driving risk dynamic early warning method based on fusion of human factors and road conditions.

[0013] Based on the above aspects, by collecting dynamic state data of drivers and security guards and real-time state data of roads during driving, a human factor state dynamic evolution sequence and a road condition real-time change sequence are generated accordingly, which can present the change trend of human factors and road conditions over time. By analyzing the interactive influence relationship between the two and establishing a human factor-road condition bidirectional correlation model, a risk trigger condition is extracted and a driving risk conduction path set is identified, path features of the driving risk conduction paths are extracted, and a risk influence range is determined, so that the risk assessment is more detailed and comprehensive. The finally generated driving risk dynamic early warning instruction containing a risk conduction direction and an influence range description can timely and accurately send early warning information to a driving control terminal to trigger a warning prompt operation, provide effective risk early warning for drivers, and thus effectively reduce driving risks. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is an execution flow schematic diagram of the driving risk dynamic early warning method based on fusion of human factors and road conditions provided by the embodiment of the present application.

[0015] Figure 2is a schematic diagram of exemplary hardware and software components of a driving risk dynamic early warning system based on human factors and road conditions fusion provided by an embodiment of the present application. DETAILED DESCRIPTION

[0016] The present application will be described in detail below with reference to the accompanying drawings, Figure 1 is a flowchart of a driving risk dynamic early warning method based on human factors and road conditions fusion provided by an embodiment of the present application, which will be described in detail below.

[0017] Step S110: Collecting dynamic state data of the driver, dynamic state data of the security guard and real-time state data of the road during driving, generating a human factor state dynamic evolution sequence based on the collected dynamic state data of the driver and the dynamic state data of the security guard, and generating a road condition real-time change sequence based on the collected real-time state data of the road.

[0018] In this embodiment, a long-distance freight vehicle driving on a highway at night is taken as an example for illustration. In this scenario, the vehicle needs to maintain high-speed driving for a long time, and the driver is prone to fatigue and other state changes. The interaction between the security guard and the driver and the real-time changes of the road conditions can all affect the driving safety, so relevant data needs to be collected for risk early warning.

[0019] Step S111: Collecting facial dynamic image data of the driver by the image collection device installed in the driver's seat at a preset time interval, extracting eye state data and facial expression data of the driver from the collected facial dynamic image data of the driver, the eye state data recording the frequency of the driver's eyelid closure and the eye movement trajectory, and the facial expression data recording the change amplitude of the driver's mouth corner and the facial muscle contraction state.

[0020] For example, a high-definition image acquisition device is installed above the instrument panel in front of the driver's seat of a long-distance freight vehicle. The device uses a wide-angle lens to capture the driver's face region completely. The preset time interval is set to capture one frame of image every 0.5 seconds to ensure that the subtle facial dynamic changes of the driver can be captured. When driving at night, the image acquisition device automatically turns on the infrared light supplement function to ensure that the facial image can still be clearly captured in low light environments. After the facial dynamic image data is collected, image preprocessing techniques are used to denoise, grayscale, and locate the face region. For eye state data extraction, a feature point detection-based method is used to further determine the positions of the eyes in the located face region. For eyelid closure frequency, the opening and closing degrees of the eyelids in multiple consecutive images are analyzed to calculate the eyelid closure frequency. When the opening and closing degree of the eyelids is lower than the preset threshold, it is determined as one closure, and the number of closures in a unit of time is the eyelid closure frequency. The extraction of eye movement trajectory is achieved by tracking the position changes of the pupil center in the image. The coordinates of the pupil center in each frame of image are connected in time sequence to form the eye movement trajectory. For facial expression data, feature point detection is also used to extract the feature points of the mouth corners and facial key muscle groups. The mouth corner change amplitude is determined by calculating the displacement of the mouth corner feature points in the vertical and horizontal directions, and the facial muscle contraction state is determined by analyzing the texture changes and grayscale value distribution of the facial muscle region, such as the change in the grayscale value of the inter-brow muscle region when frowning. The above changes are detected to record the facial muscle contraction state.

[0021] Step S112: The physiological dynamic data of the driver is synchronously collected by the physiological sensing device worn on the driver's body. The physiological dynamic data records the heart rate change data, breathing frequency data, and skin electric response data of the driver. The collected eye state data and facial expression data of the driver are integrated with the physiological dynamic data to obtain the dynamic state data of the driver.

[0022] The physiological sensing device worn by the driver includes a heart rate sensor integrated in the steering wheel grip, a respiration rate sensor installed on the seat back, and a skin conductance sensor worn on the wrist. The sensors are wirelessly connected to the central data processing unit of the vehicle through Bluetooth, achieving synchronous data transmission. The sampling frequency is consistent with the image acquisition device, both being once every 0.5 seconds to ensure time synchronization of the data. The heart rate sensor uses photoelectric sensing technology to obtain heart rate data by detecting the change in light absorption by hemoglobin in the blood. The heart rate variation data records the interval time of each heartbeat and the change trend of the number of heartbeats per unit time. The respiration rate sensor detects the driver's breathing action by sensing the pressure change of the seat back. When the driver inhales, the body will slightly recline, increasing the pressure on the seat back. When exhaling, the pressure decreases. The respiration rate is calculated by capturing the cycle of the above pressure change, and the change in breathing depth is recorded, such as the greater the pressure change amplitude, the deeper the breathing. The skin conductance sensor reflects the emotional changes of the driver by measuring the change in skin surface resistance. When the driver is in a state of tension, anxiety, and other emotional states, the skin sweat glands will increase secretion, causing the skin resistance to decrease. The sensor converts the above resistance change into an electrical signal for recording. In the data integration stage, the eye state data, facial expression data, and physiological dynamic data are aligned according to the time stamp to ensure that the data at the same time point can be matched. Then, feature extraction and standardization processing are performed on the data to combine the eyelid closure frequency, eye movement trajectory features, mouth corner change amplitude, facial muscle contraction state features, heart rate variation features, respiration rate features, and skin conductance features into the dynamic state data of the driver, where each dimension corresponds to a specific feature parameter.

[0023] Step S113: Collect the voice interaction data and action posture data of the escort through the audio acquisition device and image acquisition device installed in the vehicle cabin. The voice interaction data records the content and tone of the voice communication between the escort and the driver, and the action posture data records the amplitude and direction of the escort's body movements. The collected voice interaction data and action posture data of the escort are integrated to obtain the dynamic state data of the escort.

[0024] In the long-distance freight vehicle, a 360-degree rotating image acquisition device is installed near the top of the driver's seat, which can cover the entire activity area of the driver. Two high-sensitivity audio acquisition devices are installed in the vehicle, one near the driver's seat and the other near the driver's seat, to clearly capture the voice interaction between the two. The audio acquisition device uses a noise reduction microphone that can effectively filter out engine noise, wind noise and other environmental noise during vehicle operation. The voice interaction data is continuously collected, with a sampling frequency of 16 kHz and a quantization bit number of 16 bits to ensure the quality of the voice signal. For voice exchange content recording, the collected voice signal is converted into text information through voice recognition technology, and information such as voice signal duration and pause interval is recorded. The recording of voice tone changes is achieved by analyzing the frequency, amplitude and speed of the voice signal, for example, when the driver is emotional, the frequency and amplitude of the voice will increase, and the speed will increase. Action posture data is collected through image acquisition equipment, using a similar feature point detection method as the driver's face image acquisition, extracting key feature points of the driver's limbs such as head, shoulder, elbow, wrist, hip, knee and ankle. The amplitude of the limb movement is determined by calculating the distance change between the feature points of the adjacent frames of the image, for example, when the arm swings, the greater the displacement of the elbow feature point, the greater the amplitude of the limb movement. The direction of the action is determined by calculating the angle between the vector formed by the feature points of the limbs and the vehicle coordinate system, for example, when the driver points his arm forward, the angle between the vector formed by the arm feature points and the vehicle driving direction is small. When integrating data, the voice interaction data and action posture data are aligned according to the time stamp. Key words, tone features, etc. are extracted from the voice interaction data, and action amplitude features, action direction features, etc. are extracted from the action posture data. Then the above features are combined into the dynamic state data of the driver, containing both voice and action features.

[0025] Step S114: The collected dynamic state data of the driver is processed in time sequence, each segment of data corresponds to a time window, the core features of the dynamic state data of the driver in each time window are extracted, and the driver state node is formed; the collected dynamic state data of the driver is processed in the same time window, the core features of the dynamic state data of the driver in each time window are extracted, and the driver state node is formed.

[0026] Step S1141: Determine the length of the time window, the length of the time window is determined according to the vehicle speed, the faster the speed value, the shorter the time window value, the slower the speed value, the longer the time window value.

[0027] The driving speed of the vehicle is obtained through the CAN bus of the vehicle and transmitted to the central data processing unit in real time. The determination of the time window length adopts a dynamic adjustment mechanism, and the basic time window length is set to 5 minutes. When the driving speed of the vehicle exceeds 100 km / h, the time window length is shortened to 3 minutes. When the driving speed is between 60-100 km / h, the basic length of 5 minutes is maintained. When the driving speed is lower than 60 km / h, the time window length is extended to 8 minutes. The reason for such setting is that when driving at high speed, the road conditions change quickly, and the state change of the driver needs to be analyzed more frequently. When driving at low speed, the state change is relatively slow, and the time window can be appropriately extended to reduce the data processing amount. For example, in the night driving scene of the expressway in the embodiment, the driving speed of the vehicle usually remains at 90-110 km / h, and therefore the time window length is dynamically adjusted between 3-5 minutes.

[0028] Step S1142: The dynamic state data of the driver collected is processed in segments according to the determined time window length from the start time of collection, and a plurality of segments of dynamic state data of the driver are obtained, each segment of dynamic state data of the driver corresponding to a time window identifier.

[0029] The time segments are divided in sequence from the start time of collection according to the determined time window length. Each time window corresponds to a unique time window identifier, which contains start time and end time information. For example, the start time of the first time window is the time when the vehicle starts to collect data after starting, and the end time is the start time plus the time window length. The start time of the second time window is the end time of the first time window, and so on. In the data segmentation process, the central data processing unit will distribute the dynamic state data of the driver to the corresponding time window according to the time stamp, to ensure that the data in each time window is continuous and complete. For the data at the boundary of the time window, if the time stamp of a certain data is exactly at the junction of two time windows, it will be assigned to the latter time window.

[0030] Step S1143: The eye state data in each segment of dynamic state data of the driver is statistically analyzed, and the core features of the dynamic state data of the driver in the time window are calculated to form a driver state node.

[0031] Step S11431: The eye state data in each segment of dynamic state data of the driver is statistically analyzed, and the average value of the eyelid closure frequency and the coverage range of the eyeball rotation trajectory in the time window are calculated, and the average value and the coverage range are taken as the core features of the eye state.

[0032] In each time window, for the frequency of eyelid closure, the number of eyelid closures in the window is added up and then divided by the length of the time window to obtain the average of the frequency of eyelid closure. For example, in a 5-minute time window, 15 eyelid closures are detected in total, and the average of the frequency of eyelid closure is 3 times per minute. The coverage of the eye rotation trajectory is determined by calculating the maximum displacement of the eye rotation trajectory in the horizontal and vertical directions, multiplying the maximum displacement in the horizontal direction and the maximum displacement in the vertical direction to obtain an area value, which represents the coverage of the eye rotation trajectory. The larger the area value, the wider the range of eye rotation.

[0033] Step S11432: Analyzing the facial expression data in the section of driver dynamic state segmentation data, extracting the maximum value and the minimum value of the change amplitude of the mouth corner, and the duration of the facial muscle contraction state, and taking the maximum value, the minimum value and the duration as the core features of the facial expression.

[0034] For the change amplitude of the mouth corner, in the time window, the displacement of the mouth corner feature point is calculated frame by frame, and then the maximum value and the minimum value are found. The maximum value of the change amplitude of the mouth corner represents the maximum degree of upward or downward movement of the mouth corner in the time window, and the minimum value represents the maximum degree of movement in the opposite direction. The duration of the facial muscle contraction state is calculated by detecting the time interval from the occurrence to the end of the facial muscle contraction state. When the facial muscle contraction state feature exceeds the preset threshold, the timing starts, and stops when the feature value returns to below the threshold. This time is the duration of the facial muscle contraction state. If there are multiple muscle contractions in the time window, the duration of each contraction is added to obtain the total duration.

[0035] Step S11433: Statistics of the physiological dynamic data in the section of driver dynamic state segmentation data, calculating the fluctuation amplitude of the heart rate change data, the average value of the respiratory frequency data, and the peak value of the skin electric response data, and taking the fluctuation amplitude, the average value and the peak value as the core features of the physiological dynamics.

[0036] The fluctuation amplitude of the heart rate change data is realized by calculating the standard deviation of the heart rate data. The larger the standard deviation, the greater the heart rate fluctuation, and the more unstable the physiological state of the driver. The average value of the respiratory frequency data is obtained by adding up all the respiratory frequency values in the time window and then dividing by the number of data. The peak value of the skin electric response data is the maximum amplitude of the skin electric response signal in the time window, which reflects the maximum emotional stimulus degree of the driver in the time window.

[0037] Step S11434: The eye state core features, facial expression core features, and physiological dynamic core features are respectively standardized, and the standardized eye state core features, facial expression core features, and physiological dynamic core features are integrated into a feature vector to form a driver state node corresponding to the time window. The driver state node includes a time window identifier and a corresponding feature vector.

[0038] The standardization processing adopts a Z-score standardization method. For each core feature, the value is subtracted from the average value of the feature in the historical data, and then divided by the standard deviation of the feature, so that the standardized feature value conforms to a normal distribution with a mean of 0 and a standard deviation of 1. For example, for the average value of the eyelid closure frequency, assuming that the historical average value is 2 times per minute, the standard deviation is 1 time, and the average value in the current time window is 3 times, the standardized value is (3-2) / 1=1. After completing the standardization processing of all core features, the eye state core features (including two standardized features of the average value of the eyelid closure frequency and the coverage range of the eyeball rotation trajectory), the facial expression core features (including three standardized features of the maximum value and the minimum value of the mouth corner change amplitude, and the duration of the facial muscle contraction state), and the physiological dynamic core features (including three standardized features of the fluctuation amplitude of the heart rate variation data, the average value of the respiration frequency data, and the peak value of the skin electric response data) are arranged in order to form an 8-dimensional feature vector. The feature vector is associated with the corresponding time window identifier to jointly constitute a driver state node.

[0039] Step S1144: The collected escort dynamic state data is segmented using the same time window length to obtain multiple segments of escort dynamic state segment data. Each segment of escort dynamic state segment data corresponds to the same time window identifier as the driver state node.

[0040] The escort dynamic state data is segmented according to the determined time window length and starting from the collection start time, in the same way as the segmentation of the driver dynamic state data. Since the escort dynamic state data and the driver dynamic state data are collected synchronously, the same time window length and time window identifier are used to ensure that the driver state node and the escort state node in the same time window can be accurately associated. For example, the driver state node with a time window identifier of 19:00-19:05 corresponds to the escort dynamic state segment data with the same time window identifier.

[0041] Step S1145: The voice interaction data in each segment of escort dynamic state segment data is analyzed to extract the number of keywords in the voice communication content and the number of voice tone changes, and the number of keywords and the number of tone changes are taken as voice interaction core features.

[0042] In the analysis of the voice interaction data, first, the voice signal is subjected to voice recognition, and converted into text information. Then, through keyword matching technology, keywords related to driving safety, such as "attention", "slow down", "obstacle", "fatigue", etc. are searched for in the text information, and the total number of occurrences of these keywords is counted as a keyword quantity feature. For the number of times of voice tone changes, the frequency and amplitude changes of the voice signal are analyzed, and when the amount of change in frequency or amplitude exceeds a preset threshold, it is determined that there is one voice tone change, and the total number of voice tone changes in the time window is counted. For example, in the sentence "there is construction ahead, please pay attention to slow down" spoken by the escort, "attention" and "slow down" are keywords that will be counted in the keyword quantity, and at the same time, because the tone of the sentence may be higher than normal communication, it will be determined as one voice tone change.

[0043] Step S1146: Analyze the action posture data in the escort dynamic state segment data, calculate the average value of the limb action amplitude and the angle between the action pointing direction and the vehicle driving direction, and take the average value and the angle as the action posture core features.

[0044] For the limb action amplitude, the average value of the displacement of the escort limb feature points in the time window is calculated. The displacement of each limb feature point in each frame of image is added and then divided by the total number of frames to obtain the average displacement of the feature point, and then the maximum value of the average displacement of all limb feature points is taken as the average value of the limb action amplitude. The determination of the action pointing direction is realized by comparing the direction vector of the limb action with the vehicle driving direction vector. The vehicle driving direction vector is a preset reference direction (such as 0 degrees for the front of the vehicle), and the angle between the action pointing direction and the vehicle driving direction is determined by calculating the angle between the limb action direction vector and the reference direction. For example, when the escort stretches out his arm to point to the right side of the vehicle, the angle between the action pointing direction and the vehicle driving direction is 90 degrees.

[0045] Step S1147: Standardize the voice interaction core features and the action posture core features respectively, integrate the standardized voice interaction core features and the action posture core features into a feature vector, form an escort state node corresponding to the time window, and the escort state node contains the same time window identifier and corresponding feature vector as the driver state node.

[0046] The Z-score standardization method is also used to standardize the core features of voice interaction (number of keywords and number of voice tone changes) and the core features of action posture (average value of body movement amplitude and angle between action pointing direction and vehicle driving direction). After standardization, the four features are arranged in order to form a 4-dimensional feature vector. The feature vector is associated with the corresponding time window identifier to form the escort state node, and the time window identifier is completely consistent with the time window identifier of the driver state node in the same time window.

[0047] Step S115: associate and bind the driver state node and the escort state node corresponding to the same time window to form a human factor state node, and sequentially connect all the human factor state nodes in the order of time windows to generate a human factor state dynamic evolution sequence.

[0048] In each time window, the driver state node and the escort state node with the same time window identifier are associated and bound. The association and binding method is to splice the feature vectors of the two nodes to form a new 12-dimensional feature vector (8-dimensional driver state feature vector + 4-dimensional escort state feature vector), which is the core data of the human factor state node corresponding to the time window. At the same time, the human factor state node also contains the time window identifier and the original feature information of the driver state node and the escort state node, so that the specific human factor data source can be traced back during subsequent analysis. In the order of time windows, all human factor state nodes are arranged in order, and each node is connected in time with the previous and next nodes through the time window identifier, thereby generating a human factor state dynamic evolution sequence. The human factor state dynamic evolution sequence reflects the changes of the state of the driver and the escort over time during the entire driving process. For example, in the long-distance freight vehicle night driving scenario of the embodiment, from the start of the vehicle, the human factor state node of the first time window reflects the initial state of the driver and the preparation state of the escort, and as the driving time increases, the subsequent human factor state nodes gradually reflect the fatigue signs of the driver and the corresponding reminder actions of the escort, etc.

[0049] Step S116: through the road image acquisition device installed outside the vehicle, dynamic image data of the road surface is collected, and cover state data of the road surface and damage state data of the road surface are extracted from the dynamic image data of the road surface. The cover state data records the type and coverage range of the road surface cover, and the damage state data of the road surface records the distribution of the road surface cracks and the position of the potholes.

[0050] The road image acquisition devices are installed on the front bumper and the rearview mirror positions of the vehicle. The front device mainly acquires the road surface image in front of the vehicle, and the device at the rearview mirror position acquires the road surface image on both sides of the vehicle. The three devices can comprehensively acquire the road surface conditions on the driving path of the vehicle. The image acquisition frequency is 10 frames per second, and the resolution is 1920x1080 pixels, to ensure that the details of the road surface can be clearly captured. The acquired road surface dynamic image data is first image stitched to fuse the images acquired by different devices into a complete road surface panoramic image. Then, the road surface region is separated from the background through image segmentation technology. For the cover state data extraction of the road surface, an image recognition model based on deep learning is used. This model is trained with a large number of different types of cover images and can identify common road surface cover types such as water, snow, oil stains, and fallen leaves. After identifying the cover type, the cover range is determined by calculating the pixel proportion of the cover in the road surface area. The larger the pixel proportion, the wider the coverage. For the damage state data of the road surface, a deep learning image recognition model is also used. This model is specifically trained for road crack and pothole damage types. The distribution of cracks is represented by identifying the coordinate positions of crack pixels and connecting them into lines. The length and width information of the cracks are also recorded. The position of the pothole is determined by locating the center coordinates of the pothole area, and the area size of the pothole is recorded.

[0051] Step S117: Collect the environmental dynamic data around the road through the environmental sensing devices installed on the vehicle. The environmental dynamic data around the road records the moving track of obstacles around the road, the light change around the road, and the precipitation condition around the road.

[0052] The environmental sensing devices installed on the vehicle include laser radar, millimeter wave radar, light sensor, and precipitation sensor. The laser radar and millimeter wave radar are installed on the front and corners of the vehicle to detect obstacles around the road. The laser radar can provide high-precision three-dimensional point cloud data. By analyzing the continuous frame point cloud data, the position change of the obstacle can be tracked, and thus the moving track of the obstacle can be obtained. The millimeter wave radar has strong anti-interference ability and can effectively detect obstacles in adverse weather conditions. The data fusion of the two can improve the accuracy and reliability of obstacle detection. The light change around the road is collected by the light sensor installed on the top of the vehicle. This sensor can measure the environmental light intensity in real time with a sampling frequency of once per minute and record the change trend of the light intensity, such as gradual increase, gradual decrease, or sudden change, etc. The precipitation condition is collected by the precipitation sensor installed on the windshield of the vehicle. This sensor detects the scattering of infrared light by raindrops to determine the presence or absence of precipitation and the precipitation intensity, such as light rain, moderate rain, heavy rain, etc., and records the start and end times of the precipitation.

[0053] Step S118: Collecting road traffic dynamic data through the positioning device and the traffic information receiving device of the vehicle, the road traffic dynamic data recording the lane occupancy of the road, the driving speed of the adjacent vehicles, and the traffic signal change of the road.

[0054] The positioning device of the vehicle adopts a global navigation satellite system receiver, which can obtain the position coordinates of the vehicle in real time. Combined with electronic map data, the current road and lane information of the vehicle can be determined. By analyzing the position and speed information of the front and rear vehicles in the same lane, the lane occupancy can be calculated, such as the vehicle density and vehicle spacing of the current lane. The driving speed of the adjacent vehicles is obtained through inter-vehicle communication technology. Vehicles regularly exchange their driving speed, position, and driving direction information. The central data processing unit filters the received information and extracts the driving speed data of the vehicles in the adjacent lanes (left, right, and within a certain distance). The traffic signal change of the road is received by the traffic information receiving device, which receives real-time traffic signal information published by the traffic management department. This device can receive the state information of the traffic signal along the road, including the switching time and countdown information of red, green, and yellow lights. At the same time, when the vehicle approaches an intersection, the positioning device will combine the intersection position information in the electronic map to obtain the traffic signal change of the intersection in advance.

[0055] Step S119: Segmenting the collected road surface cover state data, road surface damage state data, road surrounding environment dynamic data, and road traffic dynamic data in time sequence, ensuring that the time window of the road condition node is consistent with the time window of the human factor state node, and extracting the core features of the above data in each time window to form a road condition node.

[0056] The same as the setting of the time window of the human factor state node, segmenting the various dynamic data related to the road in time sequence ensures that the time window of the road condition node is completely consistent with the time window of the corresponding human factor state node, so as to analyze the interaction between the two in the subsequent analysis. In each time window, the average value of the cover type (such as water accumulation, snow accumulation, etc.) and the coverage range of the road surface cover state data are extracted as the core features; the average length of the crack and the average area of the pothole of the road surface damage state data are extracted as the core features; the maximum moving speed of the obstacle, the average change rate of the light intensity, and the precipitation intensity level of the road surrounding environment dynamic data are extracted as the core features; the lane occupancy rate (quantitative indicator of lane occupancy), the average driving speed of the adjacent vehicles, and the frequency of traffic signal change of the road traffic dynamic data are extracted as the core features. After standardizing the above core features, they are integrated into a multi-dimensional feature vector, which together with the corresponding time window identifier constitutes a road condition node.

[0057] Step S120: Analyze the interaction influence relationship between each state node in the human factor state dynamic evolution sequence and each road condition node in the road condition real-time change sequence, and establish a human factor-road condition bidirectional correlation model.

[0058] Step S121: Extract all human factor state nodes from the human factor state dynamic evolution sequence, each of which contains a driver state sub-node and a security guard state sub-node, and extract all road condition nodes from the road condition real-time change sequence, each of which contains a road surface state sub-node, a road surrounding environment state sub-node, and a road traffic state sub-node.

[0059] The human factor state dynamic evolution sequence is stored in the form of a linked list, and all human factor state nodes can be extracted in turn by traversing the linked list. Each extracted human factor state node is separated into a driver state sub-node and a security guard state sub-node according to the driver state feature vector and the security guard state feature vector stored therein. The driver state sub-node contains core features such as the driver's eye, facial expression, and physiological dynamics, and the security guard state sub-node contains core features such as the security guard's speech interaction and action posture. Similarly, the road condition real-time change sequence is also stored in the form of a linked list, and all road condition nodes are extracted by traversing the linked list. Each road condition node is separated into a road surface state sub-node (containing cover and damage state features), a road surrounding environment state sub-node (containing obstacle, lighting, and precipitation features), and a road traffic state sub-node (containing lane occupancy, adjacent vehicle speed, and traffic signal features) according to the composition of the core feature vector.

[0060] Step S122: Pair each human factor state node with the road condition node corresponding to the same time window to form multiple groups of human factor-road condition node pairs, each of which contains a human factor state node and a corresponding road condition node.

[0061] Since both the human factor state node and the road condition node contain a time window identifier, the human factor state node and the road condition node within the same time window are paired by matching the time window identifier. For example, a human factor state node with a time window identifier of 19:00-19:05 is paired with a road condition node having the same identifier to form a human factor-road condition node pair. During the pairing process, it is necessary to check whether there is a mismatch in the time window identifier. If there is a human factor state node or a road condition node in a certain time window, the node is marked as an abnormal node and does not participate in the subsequent interaction influence relationship analysis, which will be processed later. After pairing, all human factor-road condition node pairs are arranged in chronological order to form a new sequence.

[0062] Step S123: Feature disassembly is performed on the human factor state node and the road condition node in each group of human factor and road condition node pairs, and driving person operation tendency features, driving person fatigue features, and escort prompt features are extracted from the human factor state node, and road resistance features, road line-of-sight interference features, and road traffic conflict features are extracted from the road condition node.

[0063] For feature disassembly of the human factor state node, features related to operation tendency are extracted from the feature vector of the driving person state sub-node, such as steering operation frequency, braking operation frequency, and acceleration operation frequency, which reflect the driving operation habits and tendency of the driving person. Driving person fatigue features are extracted from eye state core features (such as the average value of eyelid closure frequency) and physiological dynamic core features (such as the fluctuation amplitude of heart rate change data and the average value of breathing frequency data), which can reflect the fatigue degree of the driving person. Escort prompt features are extracted from voice interaction core features (such as the number of keywords and the number of voice tone changes) and action posture core features (such as the average value of limb movement amplitude and the included angle between the action pointing direction and the vehicle driving direction) of the escort state sub-node, which reflect the prompting behavior and effect of the escort on the driving person. For feature disassembly of the road condition node, road resistance features are extracted from road surface state sub-node cover data (such as cover type affecting friction coefficient) and damage state data (such as potholes increasing driving resistance), which reflect the road surface resistance to vehicle driving. Road line-of-sight interference features are extracted from road surrounding environment state sub-node light change data (such as strong light or backlight affecting line of sight), precipitation condition data (such as heavy rain blurring line of sight), and obstacle blocking data, which reflect the interference degree of the line of sight of the driving person. Road traffic conflict features are extracted from road traffic state sub-node lane occupancy (such as high lane occupancy rate leading to conflict), adjacent vehicle driving speed (such as large speed difference leading to rear-end collision), and traffic signal change (such as sudden signal change leading to red light running), which reflect the potential conflict risk in the road traffic process.

[0064] Step S124: The interaction influence degree of the driving person operation tendency features and the road resistance features in the same human factor and road condition node pair is calculated, the interaction influence degree of the driving person fatigue features and the road line-of-sight interference features is calculated, and the interaction influence degree of the escort prompt features and the road traffic conflict features is calculated.

[0065] Step S1241: The driving person operation tendency features are extracted from the human factor state node in the same human factor and road condition node pair, and the driving person operation tendency features include steering operation frequency, braking operation frequency, and acceleration operation frequency; the road resistance features are extracted from the road condition node in the node pair, and the road resistance features include road slope, road surface friction coefficient, and road flatness.

[0066] In the same person node pair, the driver operation tendency feature is extracted from the specific dimension of the driver state sub-node. The steering operation frequency is determined by analyzing the number of steering wheel angle changes of the driver within a time window, the braking operation frequency is calculated by the number of times the brake pedal is stepped on, and the acceleration operation frequency is calculated by the number of times the accelerator pedal is stepped on. The above operation frequency data has been recorded in the driver dynamic state data collection, and after feature extraction and standardization, it is stored in the feature vector of the driver state sub-node. The extraction of the road resistance feature, the road slope is measured by the vehicle's inclination sensor, which is installed on the vehicle chassis and can detect the longitudinal inclination angle of the vehicle in real time, and then converted into the road slope. The road surface friction coefficient is estimated according to the road surface covering type and the damage state, for example, the friction coefficient of dry asphalt pavement is higher, the friction coefficient of water or snow covered pavement is lower, and cracks and potholes also reduce the friction coefficient. The road roughness is measured by the vibration sensor of the vehicle suspension system, the greater the vibration, the worse the road roughness, the greater the driving resistance, and the above road resistance features are also stored in the feature vector of the road condition node after standardization.

[0067] Step S1242: Establish an interaction matrix of the driver operation tendency feature and the road resistance feature. The rows of the interaction matrix represent the parameters of the driver operation tendency feature, and the columns represent the parameters of the road resistance feature. Each element in the matrix is used to store the interaction influence value of the corresponding parameter pair.

[0068] The interaction matrix is a 3x3 matrix, with rows corresponding to steering operation frequency, braking operation frequency, and acceleration operation frequency, and columns corresponding to road slope, road surface friction coefficient, and road roughness. Each element in the matrix, such as the element in the ith row and jth column, represents the interaction influence value between the ith parameter in the driver operation tendency feature and the jth parameter in the road resistance feature.

[0069] Step S1243: Based on historical driving data, calculate the correlation coefficient between the standardized steering operation frequency and the standardized road slope as the interaction influence value; calculate the correlation coefficient between the standardized braking operation frequency and the standardized road surface friction coefficient as the interaction influence value; calculate the correlation coefficient between the standardized acceleration operation frequency and the standardized road roughness as the interaction influence value.

[0070] By analyzing a large amount of historical driving data, the statistical relationship between the driver operation tendency characteristic parameters and the road resistance characteristic parameters is established. For the steering operation frequency and the road slope, in the historical data, when the road slope increases, the driver usually increases the steering operation to keep the vehicle driving in the lane, so there is a certain positive correlation between the two. By calculating the covariance of the standardized steering operation frequency and the standardized road slope, and then dividing by the product of the standard deviations of the two, the correlation coefficient can be obtained, which is the corresponding element value of the steering operation frequency and the road slope in the interaction matrix. Similarly, the braking operation frequency and the road surface friction coefficient, when the road surface friction coefficient decreases, the driver needs to brake more frequently to ensure the safe vehicle distance, and the two are usually negatively correlated, and the correlation coefficient is calculated as the corresponding element value. The acceleration operation frequency and the road flatness, the worse the road flatness, the less the driver's acceleration operation, and the two are positively correlated, and the correlation coefficient is calculated as the corresponding element value.

[0071] Step S1244: normalize all interaction influence values in the interaction matrix to make all values in the same numerical interval, and then calculate the trace of the interaction matrix, and take the trace value as the interaction influence degree of the driver operation tendency characteristics and the road resistance characteristics.

[0072] The min-max normalization method is used for normalization, and each interaction influence value in the interaction matrix is linearly mapped to the numerical interval of 0-1. The larger the normalized interaction influence value, the stronger the interaction influence between the corresponding parameter pairs. The trace of the interaction matrix is the sum of the elements on the main diagonal line of the matrix. By calculating the trace of the normalized interaction matrix, a comprehensive numerical value is obtained, which is the interaction influence degree of the driver operation tendency characteristics and the road resistance characteristics. The larger the value, the stronger the overall interaction influence between the two types of characteristics.

[0073] Step S1245: extract the driver fatigue characteristics from the same person factor state node of the road condition node pair, including the eyelid closure time length proportion, the heart rate variability coefficient, and the reaction delay time; extract the road visual interference characteristics from the road condition node of the node pair, including the illumination intensity variation amplitude, the visibility caused by precipitation, and the surrounding object shielding range.

[0074] The extraction of the driver fatigue feature, the eyelid closure time proportion is the ratio of the total eyelid closure time to the total time in the time window, which is calculated from the eye state core feature. The heart rate variation coefficient is the ratio of the standard deviation to the average value of the heart rate change data, which reflects the fluctuation of the heart rate, which is extracted from the physiological dynamic core feature. The reaction delay time is determined by analyzing the response time of the driver to a specific stimulus (such as traffic signal change, obstacle appearance), which is extracted from the driver's operation reaction data. The road visual line interference feature, the illumination intensity change amplitude is the difference between the maximum and minimum values of the illumination intensity in the time window, which is extracted from the illumination change data of the road surrounding environment state sub-node. The visibility caused by precipitation is estimated according to the precipitation intensity and type, such as low visibility in heavy rain and high visibility in light rain, which is extracted from the precipitation condition data. The surrounding object blocking range is determined by calculating the angle of the road surrounding obstacle in the driver's field of view, which is extracted from the obstacle detection data.

[0075] Step S1246: Constructing the correlation function of the driver fatigue feature and the road visual line interference feature, the input of the correlation function is the parameters of the driver fatigue feature and the parameters of the road visual line interference feature, and the output is the interaction influence degree value.

[0076] The correlation function adopts a multi-input single-output nonlinear function form, which is trained by a large amount of sample data. The sample data contains actual interaction influence degree labels under different combinations of driver fatigue feature parameters and road visual line interference feature parameters. Through machine learning algorithms such as support vector machines, neural networks, etc., the sample data is trained to obtain the parameters of the correlation function.

[0077] Step S1247: Input the standardized eyelid closure time proportion and the standardized illumination intensity change amplitude into the first submodule of the correlation function, calculate the synergistic influence value of the two, the synergistic influence value is the calculation result of the function based on historical data training, outputting a dimensionless evaluation value; input the standardized heart rate variation coefficient and the standardized visibility caused by precipitation into the second submodule of the correlation function, calculate the coupling influence value of the two, the coupling influence value is the calculation result of the function based on historical data training, outputting a dimensionless evaluation value; input the standardized reaction delay time and the standardized surrounding object blocking range into the third submodule of the correlation function, calculate the superposition influence value of the two, the superposition influence value is the calculation result of the function based on historical data training, outputting a dimensionless evaluation value.

[0078] The first submodule of the correlation function is specifically used to process the interaction between the eyelid closure time length proportion and the illumination intensity change amplitude. When the eyelid closure time length proportion of the driver is high (indicating fatigue) and the illumination intensity change amplitude is large (such as strong light alternation when overtaking at night), the synergistic effect of the two will seriously affect the driver's vision and reaction ability, and the first submodule calculates the above synergistic effect value through the trained function. The second submodule processes the interaction between the heart rate variability coefficient and the visibility caused by precipitation. Low heart rate variability coefficient indicates that the driver's autonomic nervous regulation ability is decreased (fatigue), and low visibility increases the driving difficulty, and the coupling of the two will further increase the risk, and the second submodule calculates the coupling effect value. The third submodule processes the interaction between the reaction delay time and the surrounding object blocking range. When the reaction delay time is long and the blocking range is large, the driver has difficulty in discovering and responding to danger in time, and the third submodule calculates the superimposed effect value. The functions of the above submodules are obtained by training historical data, the input is the standardized feature parameter, and the output is a dimensionless evaluation value. The larger the evaluation value, the more significant the interaction effect.

[0079] Step S1248: Weighted sum of the output values of the first submodule, the second submodule and the third submodule to obtain the interaction influence degree of the driver fatigue feature and the road visual line interference feature.

[0080] According to the importance of the output values of each submodule in the overall interaction influence, different weights are assigned to each submodule. The weight value is determined by the analytic hierarchy process or statistical analysis based on sample data. For example, if historical data shows that the synergistic effect of the eyelid closure time length proportion and the illumination intensity change amplitude contributes most to the overall risk, the weight of the first submodule is the highest. After multiplying the output values of each submodule by the corresponding weight and adding them up, the total sum is the interaction influence degree of the driver fatigue feature and the road visual line interference feature.

[0081] Step S1249: Extract the guard prompt feature from the same person's human factor state node pair, the guard prompt feature including prompt frequency, prompt accuracy and prompt response time; extract the road traffic conflict feature from the road condition node of the node pair, the road traffic conflict feature including adjacent vehicle approach speed, lane occupation conflict frequency and traffic signal change frequency.

[0082] The extraction of the escort prompt feature, the prompt frequency is the number of prompts issued by the escort in a unit of time, which is calculated comprehensively from the number of keywords and the number of tone changes in the voice interaction core feature. The prompt accuracy is evaluated by comparing the prompt content of the escort with the actual road conditions, such as prompting "there is an obstacle ahead", and the accuracy is high if there is an obstacle, otherwise the accuracy is low, which is extracted from the comparison results of voice interaction data and road condition data. The prompt response time is the time interval from the occurrence of road condition to the prompt of the escort, which is calculated from the timestamp data. The road traffic conflict feature, the adjacent vehicle approaching speed is the relative speed of the adjacent vehicle relative to the vehicle, which is calculated from the driving speed data of the adjacent vehicle. The lane occupation conflict frequency is the number of conflicts in lane use detected by the vehicle and other vehicles or obstacles in a time window, which is extracted from the lane occupation data. The traffic signal change frequency is the number of traffic signal switches in a unit of time, which is extracted from the traffic signal change data.

[0083] Step S1250: Establish an influence evaluation model of the escort prompt feature and the road traffic conflict feature, which includes an input layer, a processing layer, and an output layer. The input layer receives parameters of the two types of features, the processing layer performs interactive operations on the parameters, and the output layer outputs the degree of interaction influence.

[0084] The influence evaluation model adopts a neural network model structure, the input layer includes 6 neurons corresponding to the prompt frequency, prompt accuracy, prompt response time of the escort prompt feature and the adjacent vehicle approaching speed, lane occupation conflict frequency, traffic signal change frequency of the road traffic conflict feature. The processing layer includes two hidden layers, the first hidden layer has 12 neurons, the second hidden layer has 8 neurons, and the ReLU activation function is used. The output layer includes one neuron, which outputs the interaction influence degree value. The model is trained by the back propagation algorithm, and the escort prompt feature and road traffic conflict feature parameters in the historical data and the corresponding interaction influence degree label are used as training data to adjust the network weight and bias to minimize the output error of the model.

[0085] Step S1251: The input layer inputs the standardized prompt frequency and the standardized adjacent vehicle approaching speed into the first operation unit of the processing layer, calculates the mitigation coefficient of the prompt frequency on the adjacent vehicle approaching risk, and the mitigation coefficient is the calculation result of the function outputting dimensionless coefficient based on historical data training.

[0086] The first operation unit is a submodule in the processing layer, and adopts a logistic regression function as a calculation function. The function is trained by historical data to obtain function parameters, and the input is the normalized prompt frequency and the adjacent vehicle approach speed. When the prompt frequency is higher and the adjacent vehicle approach speed is lower, the mitigation coefficient is larger, indicating that the prompt frequency has a stronger mitigation effect on the adjacent vehicle approach risk; otherwise, the mitigation coefficient is smaller.

[0087] Step S1252: The input layer inputs the normalized prompt accuracy and the normalized lane occupation conflict frequency into the second operation unit of the processing layer, calculates the avoidance efficiency of the prompt accuracy on the lane occupation conflict, and the avoidance efficiency is the calculation result of a function outputting a dimensionless coefficient based on historical data training.

[0088] The second operation unit also adopts a logistic regression function, and the input is the normalized prompt accuracy and the lane occupation conflict frequency. The higher the prompt accuracy and the fewer the lane occupation conflict frequency, the higher the avoidance efficiency; otherwise, the avoidance efficiency is lower. The function is also trained by historical data to obtain parameters.

[0089] Step S1253: The input layer inputs the normalized prompt response time and the normalized traffic signal change frequency into the third operation unit of the processing layer, calculates the response effect of the prompt response time on the traffic signal change risk, and the response effect is the calculation result of a function outputting a dimensionless coefficient based on historical data training.

[0090] The third operation unit adopts an exponential function form, and the input is the normalized prompt response time and the traffic signal change frequency. The shorter the prompt response time and the lower the traffic signal change frequency, the better the response effect, and the larger the dimensionless coefficient output; otherwise, the response effect is worse, and the coefficient is smaller. The function parameters are determined by historical data training.

[0091] Step S1254: The processing layer comprehensively evaluates the output results of the first, second and third operation units to determine the proportion of positive and negative effects, and the output layer calculates the interactive influence degree of the escort prompt features and the road traffic conflict features according to the proportion of positive and negative effects.

[0092] The processing layer performs a weighted average on the output results (mitigation coefficients, avoidance efficiency, response effect) of the three operation units, and the weights are determined according to the importance of each output result in the comprehensive evaluation, and the weights are obtained through statistical analysis of historical data. After the comprehensive evaluation, the sum of the positive influence and the sum of the negative influence are obtained, the proportion of the positive influence is the sum of the positive influence divided by the sum of the positive and negative influence, and the proportion of the negative influence is 1 minus the proportion of the positive influence. The output layer takes the proportion of the positive influence as the interactive influence degree of the escort prompt feature and the road traffic conflict feature, the closer the value is to 1, the stronger the positive interaction, that is, the more significant the positive influence of the escort prompt feature on the road traffic conflict feature; the closer to 0, the stronger the negative interaction.

[0093] Step S125: The three calculated interactive influence degrees are respectively standardized, and the three standardized interactive influence degrees are weighted and integrated to obtain the overall correlation strength value of each group of human-terrain node pairs. In the process of weighted integration, according to the importance of each interactive influence degree under different driving scenes, the corresponding weight coefficient is allocated.

[0094] The interactive influence degrees of the driver operation tendency feature and the road resistance feature, the interactive influence degrees of the driver fatigue feature and the road visual interference feature, and the interactive influence degrees of the escort prompt feature and the road traffic conflict feature are respectively standardized, and the Z-score standardization method is also used to make each interactive influence degree value subject to a normal distribution with a mean of 0 and a standard deviation of 1. When weighted integration, the weight coefficient is dynamically adjusted according to different driving scenes. For example, in the highway driving scene, the importance of the road traffic conflict feature is high, so the weight coefficient of the interactive influence degree of the escort prompt feature and the road traffic conflict feature will be set to be larger; in the mountain road driving scene, the importance of the road resistance feature and the road visual interference feature is high, so the weight coefficient of the corresponding interactive influence degree will be set to be larger. The determination of the weight coefficient is combined by expert experience and historical accident data analysis, to ensure that the weight distribution conforms to the importance ranking of the influencing factors of the actual driving risk. The three standardized interactive influence degrees are multiplied by the corresponding weight coefficients respectively, and then added, and the sum obtained is the overall correlation strength value of each group of human-terrain node pairs, which reflects the overall correlation tightness between the human factors and the road conditions in the time window.

[0095] Step S126: According to the time window sequence, the overall correlation strength values of the human-terrain node pairs of adjacent time windows are compared, the trend of the overall correlation strength value is analyzed, the time window in which the overall correlation strength value appears mutation is identified, and the human-terrain node pair corresponding to the mutation time window is marked as the key node pair.

[0096] In the order of time windows, the difference of the overall correlation strength values of two adjacent time windows is calculated in sequence. When the absolute value of the difference exceeds the preset mutation threshold, it is determined that the overall correlation strength value has a mutation, and the time window is the mutation time window. The setting of the mutation threshold is determined according to the normal fluctuation range of the overall correlation strength value in the historical data, for example, the upper limit of the normal fluctuation range is taken as the mutation threshold. The human-cause and road condition node pairs corresponding to the mutation time window are marked as key node pairs, which reflect the time when the correlation between the human cause state and the road condition changes significantly, and are the key objects for subsequent analysis of the bidirectional influence relationship.

[0097] Step S127: Extract the human cause state feature change amount and the road condition feature change amount in all key node pairs, analyze the triggering effect of the human cause state feature change amount on the road condition feature change amount, and analyze the feedback effect of the road condition feature change amount on the human cause state feature change amount, to form a bidirectional influence relationship description.

[0098] For each key node pair, compare it with the human cause state node and the road condition node of the previous time window, calculate the change amount of each dimension feature in the human cause state feature vector (the current node feature value minus the previous node feature value), and obtain the human cause state feature change amount. Similarly, calculate the change amount of each dimension feature in the road condition feature vector, and obtain the road condition feature change amount. Analyze the triggering effect of the human cause state feature change amount on the road condition feature change amount, for example, an increase in the driver fatigue feature change amount (an increase in fatigue level) may lead to driver operation errors, and further cause an increase in the road traffic conflict feature change amount (such as an increase in the number of lane occupation conflicts). Analyze the feedback effect of the road condition feature change amount on the human cause state feature change amount, for example, an increase in the road visual interference feature change amount (a decrease in visibility) may cause the driver to be nervous, and further cause an increase in the physiological dynamic feature change amount (such as an increase in heart rate). Through detailed analysis of the triggering effect and the feedback effect, the bidirectional influence relationship between the human and the human cause state and the road condition in the key node pair is described in natural language, and a bidirectional influence relationship description is formed.

[0099] Step S128: Based on the bidirectional influence relationship descriptions of all key node pairs and the overall correlation strength values of non-key node pairs, a human-cause and road condition bidirectional correlation model containing node correlation rules, feature interaction rules, and time evolution rules is constructed, the node correlation rules define the correlation conditions of different types of human cause state nodes and road condition nodes, the feature interaction rules define the interaction modes of human cause features and road condition features, and the time evolution rules define the change mode of the correlation relationship over time.

[0100] The node association rule is constructed by statistical analysis on the human factor state node type and the road condition node type of the key node pair and the non-key node pair. For example, when the human factor state node type is “fatigue state”, the corresponding road condition node type is mostly “low line-of-sight interference road condition”, thereby defining the association condition of the “fatigue state” human factor node and the “low line-of-sight interference road condition” node. The feature interaction rule is based on the bidirectional influence relationship description of the key node pair, and the common interaction mode between the human factor feature and the road condition feature is summarized, such as “the increase of the driver fatigue feature will cause the influence of the road line-of-sight interference feature on the driving risk to be intensified”. The time evolution rule is constructed by analyzing the change trend of the overall association strength value over time. For example, if the overall association strength value gradually increases in a plurality of continuous time windows, it indicates that the association relationship between the human factor state and the road condition is gradually enhanced, and the above gradually enhanced change mode is defined as a time evolution rule. If the overall association strength value appears periodic fluctuation, it is defined as a time evolution rule of periodic change mode. The node association rule, the feature interaction rule and the time evolution rule are integrated together to form a human factor-road condition bidirectional association model. The model can be expressed in the form of production rule, which is convenient for computer storage and reasoning.

[0101] Step S130: Based on the human factor-road condition bidirectional association model, the risk trigger conditions under different state node and road condition node combinations are extracted, and a set of driving risk transmission paths is identified.

[0102] Step S131: All node association rules in the human factor-road condition bidirectional association model are extracted, and each node association rule corresponds to a combination of human factor state node type and road condition node type.

[0103] The node association rule in the human factor-road condition bidirectional association model is stored in the form of a rule base, and each rule contains two parts of premise condition and conclusion. The premise condition is the human factor state node type and the road condition node type, and the conclusion is the association relationship between them. By traversing the rule base, the premise condition part of all rules is extracted, that is, all combinations of human factor state node type and road condition node type are obtained. For example, the rule “if the human factor state node type is “fatigue state” and the road condition node type is “low friction coefficient road condition”, then the association strength between them is high”, and the corresponding combination is the human factor state node type “fatigue state” and the road condition node type “low friction coefficient road condition”.

[0104] Step S132: For each combination of human factor state node type and road condition node type, the possible human factor feature abnormality and road condition feature abnormality under the combination are analyzed. The human factor feature abnormality includes driver operation reaction delay, driver distraction, and escort prompt delay. The road condition feature abnormality includes road surface friction coefficient reduction, sudden appearance of road peripheral obstacles, and road traffic lane reduction.

[0105] For each combination of group, according to the characteristics of the human factor state node type and the road condition node type, combined with historical accident cases and risk analysis reports, the possible human factor feature abnormality and road condition feature abnormality are analyzed. For example, the combination of the human factor state node type of “fatigue state” and the road condition node type of “low friction coefficient road condition”, the possible human factor feature abnormality is the delay of the driver's operation reaction (slow reaction due to fatigue), and the road condition feature abnormality is the reduction of the road surface friction coefficient (such as water or snow on the road surface). For the combination of the human factor state node type of “attention distraction state” and the road condition node type of “high traffic signal change frequency road condition”, the possible human factor feature abnormality is the driver's attention distraction, and the road condition feature abnormality is the reduction of the road traffic lane (such as road construction leading to lane narrowing).

[0106] Step S133: combining each human factor feature abnormality with the corresponding road condition feature abnormality to form multiple abnormal combination scenarios, each abnormal combination scenario containing one human factor feature abnormality and one corresponding road condition feature abnormality.

[0107] After analyzing the human factor feature abnormality and the road condition feature abnormality under the combination of each human factor state node type and road condition node type, the two are one-to-one corresponding combination. For example, in the above combination of “fatigue state” and “low friction coefficient road condition”, the human factor feature abnormality “delay of the driver's operation reaction” is combined with the road condition feature abnormality “reduction of the road surface friction coefficient” to form the abnormal combination scenario of “delay of the driver's operation reaction and reduction of the road surface friction coefficient”. For each combination of human factor state node type and road condition node type, multiple human factor feature abnormalities and multiple road condition feature abnormalities may be generated, so multiple abnormal combination scenarios will be formed.

[0108] Step S134: trigger condition analysis is performed on each abnormal combination scenario to determine the initial trigger factor that leads to risk under the abnormal combination scenario, and to determine the sequence of subsequent feature changes caused by the initial trigger factor.

[0109] The trigger condition analysis adopts the fault tree analysis method, takes the abnormal combination scene as a top event, and finds out the initial trigger factor causing the top event through layer-by-layer decomposition. For example, for the abnormal combination scene of "driver operation reaction delay and road surface friction coefficient reduction", the top event is the occurrence of the scene. Through analysis, the possible initial trigger factors are that the driver has been driving for too long (leading to fatigue and then operation reaction delay) and that the precipitation suddenly increases (leading to road surface friction coefficient reduction). After determining the initial trigger factors, the event tree analysis method is used to analyze the sequence of the changes in human characteristics and road conditions that may be triggered after the initial trigger factors occur. For example, the sudden increase in precipitation (initial trigger factor) first leads to the reduction of road surface friction coefficient (change in road condition characteristics), and the driver is in a state of fatigue due to the long driving time (initial trigger factor). When encountering a low friction coefficient road, the driver cannot take timely braking measures due to the operation reaction delay (change in human characteristics), and thus the vehicle braking distance is extended (subsequent change in road condition characteristics or change in vehicle driving parameters).

[0110] Step S135: Based on the initial trigger factors and the sequence of characteristic changes, a risk transmission chain corresponding to each group of abnormal combination scenes is constructed, starting from the initial trigger factors, sequentially connecting the triggered human characteristic changes or road condition characteristic changes, and ending with the possible driving risk results.

[0111] For example, step S1351: Determine the type of initial trigger factor in each group of abnormal combination scenes. If the initial trigger factor belongs to a human characteristic abnormal situation, the human characteristic abnormal situation is taken as the starting node of the risk transmission chain. If the initial trigger factor belongs to a road condition characteristic abnormal situation, the road condition characteristic abnormal situation is taken as the starting node of the risk transmission chain.

[0112] In each group of abnormal combination scenes, the initial trigger factors determined in the trigger condition analysis stage are classified to determine whether they belong to human characteristic abnormal situations or road condition characteristic abnormal situations. For example, in the abnormal combination scene of "driver attention distraction and road traffic lane reduction", if the initial trigger factor is driver attention distraction (human characteristic abnormal situation), "driver attention distraction" is taken as the starting node of the risk transmission chain. If the initial trigger factor is road traffic lane reduction (road condition characteristic abnormal situation), "road traffic lane reduction" is taken as the starting node.

[0113] Step S1352: analyze the next feature change that the feature abnormality of the starting node may directly trigger. If the starting node is a human factor feature abnormality, analyze the road condition feature change type that the human factor feature abnormality may cause, or the other associated human factor feature change type that the human factor feature abnormality may cause. If the starting node is a road condition feature abnormality, analyze the human factor feature change type that the road condition feature abnormality may cause, or the other associated road condition feature change type that the road condition feature abnormality may cause.

[0114] For the scenario where the starting node is a human factor feature abnormality, such as "driver operation reaction delay", analyze the subsequent changes that the abnormality may cause. If the driver operation reaction delay, it may cause the vehicle to have insufficient safety distance from the vehicle in front (which is a vehicle driving parameter change, and can be indirectly associated with a road traffic state change in the road condition), or further cause the driver to have a nervous emotion and thus have a faster heart rate (other associated human factor feature change type). For the scenario where the starting node is a road condition feature abnormality, such as "road surface friction coefficient reduction", analyze the human factor feature changes that it may cause, such as the driver needing to adjust the steering wheel more frequently to keep the vehicle stable (operation frequency increase in the human factor feature change type), or the vehicle brake distance being extended (road traffic resistance change in the other associated road condition feature change type).

[0115] Step S1353: take the directly triggered feature change as the second node of the risk transmission chain, and record the triggering relationship between the starting node and the second node, which includes triggering delay time and triggering condition satisfaction degree.

[0116] After determining the directly triggered feature change, it is added as the second node of the risk transmission chain. For example, the starting node is "driver distraction", and the directly triggered feature change is "failure to observe traffic signal change", and the second node is "failure to observe traffic signal change". At the same time, the triggering relationship between the two nodes is recorded, the triggering delay time is the time interval range from the occurrence of the feature abnormality of the starting node to the occurrence of the feature change of the second node, and the triggering condition satisfaction degree is the level of the abnormality of the starting node that will trigger the change of the second node.

[0117] Step S1354: take the second node as the starting point, repeat the above analysis process to determine the next feature change that the second node may directly trigger, as the third node of the risk transmission chain, and record the triggering relationship between the second node and the third node.

[0118] Take the second node "traffic signal change not observed" as the starting point, analyze the next feature change it may trigger, such as "vehicle not decelerating according to traffic signal indication", and take it as the third node. Record the triggering relationship between the two, such as "the condition of vehicle not decelerating is met when the traffic signal change is not observed for a certain time and the vehicle speed is higher than a certain proportion of the speed limit on this section of road", and record the triggering delay time range.

[0119] Step S1355: Continue to repeat the above steps S1352 to S1354 until the analyzed feature change directly leads to a driving risk result, and take the driving risk result as the terminal node of the risk transmission chain.

[0120] Continue to repeat the feature change analysis and node addition process, for example, the third node "vehicle not decelerating according to traffic signal indication" may trigger "increased risk of collision with vehicles traveling transversely at the intersection" (fourth node), which further triggers "vehicle side collision" (driving risk result), at which point "vehicle side collision" is taken as the terminal node. If multiple possible branches occur during the transmission process, such as the fourth node simultaneously triggering "vehicle emergency braking leading to side slip" and "increased collision risk", two branch chains are constructed respectively, both ending at the corresponding driving risk result.

[0121] Step S1356: During the construction process, if a certain feature change node may trigger multiple subsequent feature change nodes at the same time, for each possible subsequent node, a branch transmission chain is constructed respectively, forming a multi-branch risk transmission chain structure.

[0122] When a node has multiple triggering directions, such as the road condition feature anomaly node "sudden appearance of obstacles around the road", which may simultaneously trigger "driver emergency steering" and "driver emergency braking" two human feature change nodes, take this node as the parent node, and construct two branch chains respectively: one is "sudden appearance of obstacles around the road → driver emergency steering → excessive steering angle leading to vehicle rollover", the other is "sudden appearance of obstacles around the road → driver emergency braking → insufficient braking distance leading to rear-end collision". Each branch chain independently records the triggering relationship and node information.

[0123] Step S1357: For each node in the constructed risk transmission chain, supplement the feature change parameters corresponding to the node, including the change amplitude, change duration, and feature value after the change.

[0124] For each feature change node in the chain, specific parameters are supplemented, such as the "driver eyelid closure frequency increase" node, the change amplitude is "increased by a certain percentage compared to the previous time window", the change duration is "continuous for multiple time windows", and the changed feature value is "eyelid closure frequency reaches a certain level". For the road condition feature node "lane occupancy conflict frequency increases", the change amplitude is "increased by a certain number compared to the average value of the previous period", the change duration is "continuous within the current time window", and the changed feature value is "lane occupancy conflict frequency reaches a certain number".

[0125] Step S1358: For each trigger relationship in the risk transmission chain, supplement the trigger probability information, which is based on the frequency statistics of the occurrence of the trigger relationship in historical driving data. The larger the frequency value, the larger the trigger probability value.

[0126] According to historical data statistics, trigger probability is added to each trigger relationship. For example, the trigger relationship "driver fatigue feature increases → road visual obstruction is not discovered in time", which appears multiple times in historical data, and the actual trigger of not discovering in time occupies a certain proportion, then the trigger probability is the proportion value. For the trigger relationship "road surface friction coefficient decreases → vehicle braking distance extends", if the combination appears multiple times in historical data, and the trigger of braking distance extension occupies a high proportion, then the trigger probability is the proportion value. The trigger probability is recorded in the trigger relationship in the form of percentage, which is used for weight calculation in subsequent risk assessment.

[0127] Step S1359: According to the node order and trigger relationship, the starting node, intermediate feature change node, terminal node and corresponding parameter information, probability information are connected in series to form a risk transmission chain, each risk transmission chain is represented by a structured node sequence, which contains node type, node parameter, trigger relationship, trigger probability.

[0128] All nodes in the chain are arranged in the conduction order to form a structured node sequence. Taking the "human factor triggered" chain as an example, the node sequence is represented as: [start node: type = driver fatigue feature (increased eyelid closure frequency), parameter = change amplitude, duration, feature value; trigger relationship: to intermediate node 1, delay time, condition satisfaction degree, trigger probability; intermediate node 1: type = human factor operation feature (brake operation delay), parameter = change amplitude, duration, feature value; trigger relationship: to intermediate node 2, delay time, condition satisfaction degree, trigger probability; intermediate node 2: type = road traffic feature (brake distance extension), parameter = change amplitude, duration, feature value; trigger relationship: to end node, delay time, condition satisfaction degree, trigger probability; end node: type = driving risk result (rear-end collision)]. The information of each node and trigger relationship is contained in the structured representation, which facilitates subsequent model matching and verification.

[0129] Step S136: Match all the constructed risk conduction chains with the time evolution rules in the human-traffic two-way association model, verify whether the time intervals of each link in the risk conduction chain meet the time threshold requirements in the time evolution rules, and eliminate the risk conduction chains that do not meet the time threshold requirements.

[0130] The time evolution rule in the human-traffic two-way association model defines the change pattern of the association relationship with time, which contains the time interval threshold requirements between different feature change links. For example, the time evolution rule may stipulate that "after the road surface friction coefficient decreases, the driver should make a brake response within 2 seconds, otherwise the risk will increase", and the 2 seconds here is the time interval threshold. Compare the time interval of the adjacent two links in the risk conduction chain with the corresponding time threshold in the time evolution rule. If the time interval is less than or equal to the time threshold, the chain is verified; if the time interval is greater than the time threshold, the chain does not meet the time requirement and is eliminated. For example, the time interval from "road surface friction coefficient reduction" to "driver brake operation delay" in a certain risk conduction chain is 3 seconds, while the time threshold in the time evolution rule is 2 seconds, so the chain is not verified and is eliminated.

[0131] Step S137: Supplement the missing feature change links in the verified risk conduction chain to make the risk conduction chain fully reflect the conduction process from the initial trigger factor to the driving risk result.

[0132] In constructing the risk transmission chain, there may be missing characteristic change links in the chain due to incomplete initial analysis. For example, a risk transmission chain is "driver distraction → failure to notice traffic signal change → red light running → vehicle collision", but actually, there is a characteristic change link "vehicle not decelerating" between "failure to notice traffic signal change" and "red light running". By comparing with the time evolution rule and the actual accident case, these missing links are identified and supplemented to the risk transmission chain, making the chain more complete and accurate. The supplemented chain is "driver distraction → failure to notice traffic signal change → vehicle not decelerating → red light running → vehicle collision".

[0133] Step S138: classify all the supplemented complete risk transmission chains, divide them into three groups according to the type of the initial risk transmission factor, i.e. human factor triggered, road condition triggered, and human factor and road condition jointly triggered, and the risk transmission chains in each group constitute a driving risk transmission sub-set of the type.

[0134] The human factor triggered risk transmission chain refers to the chain where the initial trigger factor is an abnormal human factor, such as "driver fatigue → delayed operation response → vehicle out of control". The road condition triggered risk transmission chain refers to the chain where the initial trigger factor is an abnormal road condition, such as "road surface with large potholes → vehicle jolt → loss of direction control → vehicle off the road". The human factor and road condition jointly triggered risk transmission chain refers to the chain where the initial trigger factor contains both an abnormal human factor and an abnormal road condition, such as "driver distraction and sudden appearance of obstacles on the road → failure to avoid obstacles → vehicle collision with obstacles". For all the supplemented complete risk transmission chains, classify them according to the type of the initial trigger factor and assign them to the corresponding driving risk transmission sub-set.

[0135] Step S139: integrate all the driving risk transmission sub-sets to form a driving risk transmission path set containing different types of risk transmission paths, each driving risk transmission path corresponds to a unique path identifier in the set, and the path identifier contains the initial factor type, the number of transmission links, and the risk result type information.

[0136] The three sets of risk transmission chains of human triggered, road triggered, and human-road triggered are integrated to form a set of risk transmission paths. Each risk transmission path is assigned a unique path identifier, which is in the form of a string code, including the starting factor type code (e.g., "R" for human triggered, "L" for road triggered, and "RL" for human-road triggered), the number of transmission links (represented by a number, e.g., "5" for 5 transmission links), and the risk result type code (e.g., "CZ" for vehicle collision and "PL" for vehicle lane deviation). For example, a risk transmission path of human triggered, 4 transmission links, and vehicle collision as the risk result, its path identifier can be "R-4-CZ".

[0137] Step S140: Path feature extraction is performed on each risk transmission path in the set of risk transmission paths, and the risk impact range corresponding to each risk transmission path is determined according to the extracted path features.

[0138] Step S141: A risk transmission path is selected from the set of risk transmission paths, and all transmission links in the risk transmission path are extracted, each transmission link corresponding to a feature change event, and the feature change event including the event type, the event occurrence time, and the human or road element affected by the event.

[0139] From the set of risk transmission paths, each risk transmission path is selected in order according to the path identifier. For the selected path, all transmission links contained therein are extracted one by one. Each transmission link corresponds to a feature change event, and the event type is divided into human feature change event and road feature change event, such as "driver operation reaction delay" as a human feature change event and "road surface friction coefficient reduction" as a road feature change event. The event occurrence time refers to the relative time of the feature change event in the entire risk transmission process, with the time of the initial triggering factor as the reference time, and the occurrence time of the subsequent event as the time offset relative to the reference time. The human or road element affected by the event refers to the human element (such as the reaction ability of the driver and the prompting behavior of the escort) or the road element (such as the road surface state and the road traffic state) directly affected by the feature change event.

[0140] Step S142: analyze the influence degree of each feature change event on the vehicle driving parameters, including driving speed, driving direction, braking distance, steering angle, determine the independent influence value of each feature change event on each vehicle driving parameter by comparing the change amplitude of the vehicle driving parameters before and after the event occurs; normalize each independent influence value of each feature change event respectively, and comprehensively calculate the normalized independent influence values to obtain a single and dimensionless parameter influence value of the feature change event.

[0141] For each feature change event, the change of vehicle driving parameters before and after the event occurs is simulated by a vehicle dynamics model. For example, for the road surface friction coefficient reduction, the change of braking distance when the vehicle brakes with the same brake pedal force before and after the friction coefficient decreases is simulated, and the increase of braking distance is the independent influence value of the event on the braking distance. For the driver operation reaction delay, the change of steering angle of the vehicle before and after the reaction delay under the same road conditions is simulated, and the deviation of the steering angle is the independent influence value of the event on the steering angle. The independent influence values of each vehicle driving parameter are normalized respectively, and the min-max normalization method is used to map them to 0-1. When comprehensively calculating, according to the importance of each vehicle driving parameter in different risk scenarios, each parameter is assigned a weight, such as in the high-speed driving scenario, the weights of braking distance and driving speed are larger, and the weights of driving direction and steering angle are relatively smaller. After multiplying the normalized independent influence values by the corresponding weights and adding them, the parameter influence value of the feature change event is obtained, and the larger the value, the greater the comprehensive influence of the event on the vehicle driving parameters.

[0142] Step S143: accumulate the parameter influence values of all the transmission links in the order of transmission to obtain the total parameter influence value of the driving risk transmission path, and in the accumulation process, weight each parameter influence value according to the position weight of the transmission link in the path, and the weight coefficient value of the front-end link of the path is smaller than that of the rear-end link.

[0143] The later the position of the transmission link in the path, the more direct the influence on the final driving risk result, and therefore the larger the weight coefficient. The position weight is determined in an exponentially increasing manner, for example, the weight coefficient of the first transmission link is 0.1, the second is 0.2, the third is 0.4, the fourth is 0.8, and so on, and the specific weight coefficient value is adjusted according to the total number of transmission links of the path to ensure that the sum of all weight coefficients is 1. Multiply the parameter influence value of each transmission link by its corresponding position weight coefficient, then add all the weighted parameter influence values to obtain the total parameter influence value of the driving risk transmission path, which reflects the overall influence of the entire risk transmission path on the vehicle driving parameters.

[0144] Step S144: Extract the conduction speed feature of the driving risk conduction path, which is obtained by calculating the average value of the time interval between adjacent conduction links. The smaller the average value of the time interval, the faster the corresponding conduction speed value of the conduction speed feature.

[0145] The time interval between adjacent conduction links refers to the time of occurrence of the event of the next conduction link minus the time of occurrence of the event of the previous conduction link. The average value of the time interval of all adjacent conduction links in the path is calculated, which is the quantitative indicator of the conduction speed feature. The smaller the average value of the time interval, the faster the conversion speed between the links in the risk conduction process, and the faster the risk development; on the contrary, the larger the average value of the time interval, the slower the risk development.

[0146] Step S145: According to the total parameter influence value and the conduction speed feature, query the preset influence range mapping table, wherein the influence range mapping table stores the risk influence range description corresponding to different total parameter influence value intervals and conduction speed intervals, and the risk influence range description includes the affected vehicle system and the affected driving area.

[0147] The influence range mapping table is pre-established according to a large amount of accident data analysis and vehicle system simulation results. In the table, a plurality of intervals (such as 0-0.3, 0.3-0.6, 0.6-1.0) are divided according to the total parameter influence value, and a plurality of intervals (such as greater than 5 seconds, 3-5 seconds, and less than 3 seconds) are divided according to the average value of the time interval of the conduction speed feature. Each combination of the total parameter influence value interval and the conduction speed interval corresponds to a risk influence range description. For example, the combination of the total parameter influence value interval 0.6-1.0 and the conduction speed interval less than 3 seconds corresponds to the risk influence range description "affecting the vehicle braking system and the steering system, and the affected driving area is the current lane and the adjacent left lane". According to the total parameter influence value and the conduction speed feature of the current driving risk conduction path, the interval combination to which it belongs is determined, and then the corresponding risk influence range description is queried from the influence range mapping table.

[0148] Step S146: Analyze the risk result type in the driving risk conduction path, determine the secondary risk type that the risk result may cause, supplement the queried risk influence range description in combination with the secondary risk type, form the risk influence range information, and the secondary risk type includes vehicle out-of-control risk, rear-end risk, and lane deviation risk.

[0149] The secondary risk type that the risk result type such as "vehicle collision" can cause includes vehicle out-of-control risk (vehicle out-of-control after collision), fuel leakage risk (tank damage caused by collision), and the like. By analyzing the characteristics of the risk result type and possible subsequent development, the secondary risk type is determined. Then, the risk impact range description queried from the impact range mapping table is supplemented according to the secondary risk type. For example, the original description is "impact vehicle braking system, steering system, and the driving area affected is the current lane and the adjacent left lane", if the risk result type is "vehicle collision" and the secondary risk type is "vehicle out-of-control risk", the supplemented risk impact range information is "impact vehicle braking system, steering system, and vehicle body structure system, and the driving area affected is the current lane, the adjacent left lane, and the road central median area, and the vehicle out-of-control risk can be caused".

[0150] Step S147: Repeat the above steps S141 to S146 to perform path feature extraction and risk impact range determination on each driving risk transmission path in the set of driving risk transmission paths, so that each driving risk path corresponds to unique risk impact range information.

[0151] According to the process of steps S141 to S146, each path in the set of driving risk transmission paths is processed in turn. Each path goes through the steps of transmission link extraction, impact degree analysis of feature change events, total parameter impact value calculation, transmission speed feature extraction, impact range mapping query, and secondary risk type supplementation, and finally obtains unique risk impact range information corresponding to each path.

[0152] Step S148: Store the path identification of each driving risk transmission path and the corresponding risk impact range information in association to form a path impact range association table, and the path impact range association table also contains the total parameter impact value and the transmission speed feature of each path.

[0153] The path impact range association table is stored in the form of a database table, which contains a path identification field, a total parameter impact value field, a transmission speed feature field, and a risk impact range information field. The path identification, total parameter impact value, transmission speed feature, and corresponding risk impact range information of each driving risk transmission path are filled into the corresponding fields in the table for associated storage. Through the association table, the risk impact range, total parameter impact value, and transmission speed feature information corresponding to any path identification can be quickly queried.

[0154] Step S150: Generate a driving risk dynamic warning instruction containing risk transmission direction and risk impact range description in combination with the risk impact range corresponding to each driving risk transmission path, and send the driving risk dynamic warning instruction to the driving control terminal to trigger the driving control terminal to perform a warning prompt operation.

[0155] Step S151: Extract the path identifier, risk impact range information, total parameter impact value, and conduction speed characteristic of all driving risk conduction paths from the path impact range association table.

[0156] Through a database query operation, the values of the path identifier, risk impact range information, total parameter impact value, and conduction speed characteristic fields of all records in the path impact range association table are read, and the above data is loaded into memory to form a data list for subsequent priority sorting and early warning instruction generation.

[0157] Step S152: Priority sort all driving risk conduction paths according to the size of the total parameter impact value. The larger the total parameter impact value, the higher the priority of the path. For paths with the same total parameter impact value, secondary sorting is performed according to the conduction speed characteristic of the conduction speed. The faster the conduction speed value, the higher the priority of the path.

[0158] A multi-key sorting algorithm is used. First, the total parameter impact value is used as the first sorting key and sorted in descending order. When the total parameter impact value is the same, the conduction speed characteristic is used as the second sorting key. The smaller the average time interval of the conduction speed characteristic, the faster the conduction speed. Therefore, the average time interval is sorted in ascending order (i.e., the faster the conduction speed, the higher the sorting position). After sorting, a sequence of driving risk conduction paths is obtained, sorted in descending order of priority.

[0159] Step S153: Select a preset number of driving risk conduction paths as core early warning paths in order of priority from high to low. The preset number is determined according to the complexity of the current driving scene.

[0160] The determination of the preset number is related to the complexity of the current driving scene. The complexity of the driving scene is evaluated by considering factors such as road type (e.g., expressway, urban road, rural road), traffic flow, weather conditions, etc. For example, in a complex scene of an expressway, heavy traffic, and rainy weather, the preset number is set to 10; in a general scene of an urban road, medium traffic, and sunny weather, the preset number is set to 5; in a simple scene of a rural road, light traffic, and overcast weather, the preset number is set to 3. In order of priority from high to low, the preset number of paths is selected from the sorted sequence of driving risk conduction paths as core early warning paths.

[0161] Step S154: Extract the risk transmission path of each core early warning path, determine the characteristic change direction of each transmission link in the risk transmission path, and concatenate the characteristic change directions in the transmission order to obtain the risk transmission direction of the core early warning path. The risk transmission direction is represented by a direction vector sequence, and each direction vector corresponds to the characteristic change direction of a transmission link.

[0162] For each core early warning path, extract all transmission links contained therein, i.e. the sequence of characteristic change events. Analyze the characteristic change direction of each characteristic change event, for example, the characteristic change direction of "road surface friction coefficient reduction" is "negative change" (friction coefficient value decreases), and the characteristic change direction of "driver operation reaction delay increase" is "positive change" (delay time increases). The characteristic change direction is represented by a direction vector, for example, "+1" represents positive change, "-1" represents negative change, and "0" represents no obvious change (in actual application, there may be more direction definitions according to the type of characteristic). Arrange the direction vectors of each transmission link in the transmission order to form a direction vector sequence, which is the risk transmission direction of the core early warning path. For example, the transmission links of a core early warning path are "road surface friction coefficient reduction → driver operation reaction delay increase → brake distance extension", and the corresponding direction vector sequence is [-1, +1, +1].

[0163] Step S155: Extract the affected vehicle system and the affected driving area from the risk impact range information corresponding to each core early warning path, and integrate the affected vehicle system and the affected driving area into a risk impact range description. The risk impact range description is expressed in a structured language, and the specific impact mode of each affected object is determined.

[0164] The risk impact range information includes the affected vehicle system (such as the braking system, the steering system, the vehicle body structure system, etc.) and the affected driving area (such as the current lane, the adjacent left lane, the road central barrier area, etc.). After extracting this information, it is expressed in a structured language, which includes objects, impact modes, impact degrees, etc. For example, the affected vehicle system is "braking system", the impact mode is "braking efficiency decline", and the impact degree is "severe"; the affected driving area is "current lane", the impact mode is "vehicle trajectory deviation", and the impact degree is "moderate". Combine the above elements to form a risk impact range description, such as "braking system: severe decline in braking efficiency; steering system: moderate decrease in steering accuracy; current lane: moderate deviation of vehicle trajectory; adjacent left lane: risk of vehicle collision".

[0165] Step S156: Integrate the path identifier, risk transmission direction, and risk impact range description of each core early warning path to form a single-path early warning information. Each single-path early warning information contains a path identifier field, a transmission direction field, and an impact range field.

[0166] The single-path early warning information adopts a JSON format data structure. The path identifier field stores a path identifier string, the transmission direction field stores a direction vector sequence (such as [-1, +1, +1]), and the impact range field stores a structured risk impact range description text. For example, a single-path early warning information may be as follows: {"path identifier": "R-4-CZ", "transmission direction": [-1, +1, +1], "impact range": "braking system: severe decline in braking effectiveness; steering system: moderate reduction in steering accuracy; current lane: moderate deviation of vehicle trajectory; adjacent left lane: risk of vehicle collision"}.

[0167] Step S157: Aggregate all single-path early warning information corresponding to the core early warning paths, add the current early warning generation time and early warning validity period information, and generate a driving risk dynamic early warning instruction. The driving risk dynamic early warning instruction is written in a format that can be parsed by a driving control terminal and contains a command header, a command body, and a command tail. The command header contains a command type identifier, the command body contains all single-path early warning information, and the command tail contains checksum information.

[0168] The current early warning generation time is the specific time when the system generates the early warning instruction, accurate to the second. The early warning validity period is determined based on the transmission speed characteristics of the risk transmission path and the current driving speed. The faster the transmission speed and the higher the driving speed, the shorter the early warning validity period, for example, set to 5-10 minutes. The driving risk dynamic early warning instruction adopts a custom binary protocol format. The command header contains command type identifiers (such as 0x01 for driving risk dynamic early warning instructions) and command length information. The command body contains the current early warning generation time, the early warning validity period, and all single-path early warning information (JSON format data after serialization processing). The command tail contains a CRC checksum for integrity verification of the received instruction by the driving control terminal.

[0169] Step S158: The generated driving risk dynamic early warning instruction is sent to the driving control terminal, so that after the driving control terminal receives the driving risk dynamic early warning instruction, the single-path early warning information in the instruction body is parsed, and according to the risk transmission direction and risk influence range description obtained by parsing, the corresponding early warning prompt mode is determined, the determined early warning prompt mode performs the early warning prompt operation, and the early warning execution time, the early warning prompt mode, the driver's response operation to the early warning are recorded to form an early warning execution log, which is used for subsequent evaluation of the early warning effect. The early warning prompt mode includes voice prompt, light prompt, seat vibration prompt, and different risk influence range descriptions correspond to different early warning prompt combinations.

[0170] The driving risk dynamic early warning instruction is sent to the driving control terminal through the CAN bus or Ethernet in the vehicle. After the driving control terminal receives the instruction, the instruction header is first parsed and CRC checked. After the checking passes, the instruction body is parsed to extract the current early warning generation time, early warning validity period and single-path early warning information. According to the risk transmission direction (direction vector sequence) and risk influence range description (structured language expression) in the single-path early warning information, the early warning prompt mode is determined. For example, when the risk influence range description contains "braking system is seriously affected" and "current lane vehicle trajectory is seriously deviated", the early warning prompt mode combination is: voice prompt (repeatedly playing "attention! The braking system efficiency is seriously decreased, the vehicle trajectory is deviated, and immediately slow down!"), light prompt (instrument panel red brake failure light and turn signal light flashing), seat vibration prompt (left and right seats alternating high-intensity vibration). After the early warning prompt operation is performed, the driving control terminal records the early warning execution time (accurate to seconds), the early warning prompt mode (specific combination and parameters of voice, light and seat vibration), the driver's response operation to the early warning (such as whether to step on the brake pedal, whether to turn the steering wheel, whether to reduce the driving speed, etc.), forms an early warning execution log, and stores it in the local storage of the terminal, so as to evaluate the early warning effect through subsequent data analysis.

[0171] In an exemplary embodiment, a driving risk dynamic early warning system based on fusion of human factors and road conditions is provided, which can be a terminal, a server, etc., and its internal structure diagram can be as follows Figure 2The shown. The driving risk dynamic early warning system based on human factors and road conditions fusion includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor is used to provide computing and control ability. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface is used to exchange information between the processor and external devices. The communication interface is used to communicate with external terminals in a wired or wireless manner. Wireless mode can be achieved through WIFI, mobile cellular network, near field communication or other technologies. The computer program is executed by the processor to implement a driving risk dynamic early warning method based on human factors and road conditions fusion. The display unit is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device can be a touch layer overlaid on the display screen, or a button, trackball or touchpad arranged on the shell of the driving risk dynamic early warning system based on human factors and road conditions fusion, or an external keyboard, touchpad or mouse, etc.

[0172] It should be noted that, in order to simplify the expression of the present disclosure and help understand one or more embodiments of the present application, in the foregoing description of embodiments of the present application, various features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A dynamic early warning method for driving risks based on the fusion of human factors and road conditions, characterized in that, The method includes: The system collects dynamic status data of drivers, escorts, and real-time road status data during the driving process. Based on the collected dynamic status data of drivers and escorts, it generates a sequence of dynamic evolution of human factors and a sequence of real-time changes in road conditions. The interaction between each state node in the dynamic evolution sequence of human factors and each road condition node in the real-time change sequence of road conditions is analyzed, and a two-way correlation model between human factors and road conditions is established. Based on the aforementioned bidirectional correlation model between human factors and road conditions, risk triggering conditions under different combinations of state nodes and road condition nodes are extracted, and a set of driving risk transmission paths is identified. For each driving risk transmission path in the set of driving risk transmission paths, path features are extracted, and the risk impact range corresponding to each driving risk transmission path is determined based on the extracted path features; Based on the risk impact range corresponding to each driving risk transmission path, a dynamic driving risk warning instruction containing a description of the risk transmission direction and risk impact range is generated, and the dynamic driving risk warning instruction is sent to the driving control terminal to trigger the driving control terminal to perform a warning prompt operation; The analysis of the interaction between each state node in the dynamic evolution sequence of human factors and each road condition node in the real-time change sequence of road conditions establishes a two-way correlation model between human factors and road conditions, including: All human-factor state nodes are extracted from the dynamic evolution sequence of human-factor state, and each human-factor state node includes a driver state sub-node and an escort state sub-node. All road condition nodes are extracted from the real-time road condition change sequence, and each road condition node includes a road surface state sub-node, a road surrounding environment state sub-node, and a road traffic state sub-node. Each human factor status node is paired with the road condition node corresponding to the same time window to form multiple pairs of human factor road condition nodes. Each pair of human factor road condition nodes includes a human factor status node and a corresponding road condition node. For each pair of human-factor and road condition nodes, feature decomposition is performed on the human-factor state nodes and road condition nodes. Driver operation tendency features, driver fatigue features, and escort prompt features are extracted from the human-factor state nodes. Road resistance features, road vision interference features, and road traffic conflict features are extracted from the road condition nodes. Calculate the degree of interaction between driver's operational tendencies and road resistance characteristics, driver fatigue characteristics and road visual interference characteristics, and escort personnel's prompting characteristics and road traffic conflict characteristics for the same person at road condition nodes. The calculated levels of the three interaction effects are standardized, and the standardized levels of the three interaction effects are weighted and integrated to obtain the overall correlation strength value of each pair of human-road condition node pairs. In the weighted integration process, corresponding weight coefficients are assigned according to the importance of each interaction effect level under different driving scenarios. By comparing the overall correlation strength values ​​of human factors and road conditions node pairs in adjacent time windows according to the order of time windows, analyzing the changing trend of the overall correlation strength values, identifying the time windows in which the overall correlation strength values ​​change abruptly, and marking the human factors and road conditions node pairs corresponding to the abrupt change time windows as key node pairs; Extract the changes in human state characteristics and road condition characteristics from all key node pairs, analyze the triggering effect of changes in human state characteristics on changes in road condition characteristics, and analyze the feedback effect of changes in road condition characteristics on changes in human state characteristics to form a description of two-way influence relationship. Based on the description of the bidirectional influence relationship of all key node pairs and the overall association strength value of non-key node pairs, a bidirectional association model of human factors and road conditions is constructed, which includes node association rules, feature interaction rules, and time evolution rules. The node association rules define the association conditions between different types of human factor state nodes and road condition nodes, the feature interaction rules define the interaction mode between human factor features and road condition features, and the time evolution rules define the change pattern of the association relationship over time.

2. The method for dynamic early warning of driving risks based on the fusion of human factors and road conditions according to claim 1, characterized in that, The process involves collecting dynamic state data of the driver, escort, and real-time road conditions during the driving process. Based on the collected dynamic state data of the driver and escort, a dynamic evolution sequence of human factors is generated; based on the collected real-time road condition data, a real-time change sequence of road conditions is generated. This includes: The driver's facial dynamic image data is collected by the image acquisition device installed in the driver's seat at preset time intervals. The driver's eye state data and facial expression data are extracted from the collected facial dynamic image data. The eye state data records the driver's eyelid closure frequency and eyeball rotation trajectory, and the facial expression data records the driver's mouth corner change range and facial muscle contraction state. By wearing physiological sensing devices on the driver, the driver's physiological dynamic data is collected synchronously. The physiological dynamic data records the driver's heart rate change data, respiratory rate data, and skin conductance response data. The collected driver's eye state data, facial expression data and physiological dynamic data are integrated to obtain the driver's dynamic state data. The audio and image acquisition devices installed in the carriage collect the escort's voice interaction data and movement posture data. The voice interaction data records the content of the escort's voice communication with the driver and the changes in voice tone. The movement posture data records the escort's body movement range and direction. The collected escort's voice interaction data and movement posture data are integrated to obtain the escort's dynamic state data. The collected dynamic status data of drivers are segmented according to time sequence, with each segment corresponding to a time window. The core features of the dynamic status data of drivers within each time window are extracted to form driver status nodes. Similarly, the collected dynamic status data of escorts are segmented according to the same time window, and the core features of the dynamic status data of escorts within each time window are extracted to form escort status nodes. The driver status node and the escort status node corresponding to the same time window are associated and bound to form a human factor status node. All human factor status nodes are connected in sequence according to the time window to generate a dynamic evolution sequence of human factor status. By using road image acquisition equipment installed on the outside of the vehicle, dynamic image data of the road surface is collected. From the dynamic image data of the road surface, data on the status of road surface coverings and data on the damage status of the road surface are extracted. The data on the status of road surface coverings records the type and coverage area of ​​the road surface coverings, while the data on the damage status of the road surface records the distribution of cracks and the location of potholes. By installing environmental sensing devices on vehicles, dynamic environmental data around the road is collected. The dynamic environmental data around the road records the movement trajectory of obstacles around the road, changes in light around the road, and precipitation around the road. By using vehicle positioning equipment and traffic information receiving equipment, dynamic traffic data of the road is collected. The dynamic traffic data of the road records the lane occupancy, the speed of adjacent vehicles, and changes in road traffic signals. The collected data on road surface cover status, road surface damage status, road surrounding environment dynamics, and road traffic dynamics are segmented according to time sequence. The segmented time window is consistent with the time window for generating human condition status nodes. The core features of the above data within each time window are extracted to form road condition nodes. All road condition nodes are connected in sequence according to the time window to generate a real-time road condition change sequence.

3. The method for dynamic early warning of driving risks based on the fusion of human factors and road conditions according to claim 1, characterized in that, Based on the aforementioned human-road-condition bidirectional correlation model, risk triggering conditions under different combinations of state nodes and road condition nodes are extracted, and a set of driving risk transmission paths is identified, including: All node association rules are extracted from the bidirectional human-factor and road condition association model. Each node association rule corresponds to a combination of human-factor state node type and road condition node type. For each combination of human factor state node type and road condition node type, analyze the possible abnormal human factor characteristics and abnormal road condition characteristics under this combination. Abnormal human factor characteristics include delayed driver operation response, driver inattention, and untimely prompts from escort personnel. Abnormal road condition characteristics include reduced road surface friction coefficient, sudden appearance of obstacles around the road, and reduction of road traffic lanes. Each human factor characteristic anomaly is combined with the corresponding road condition characteristic anomaly to form multiple sets of anomaly combination scenarios. Each set of anomaly combination scenarios includes one human factor characteristic anomaly and one corresponding road condition characteristic anomaly. For each abnormal combination scenario, trigger condition analysis is performed to determine the initial triggering factors that lead to the risk under the abnormal combination scenario, and at the same time, the order in which the initial triggering factors cause subsequent characteristic changes is determined. Based on the initial triggering factors and the sequence of feature changes, a risk transmission chain is constructed for each abnormal combination scenario. The risk transmission chain starts from the initial triggering factor, connects sequentially to the triggered human factor feature changes or road condition feature changes, and ends at the possible driving risk outcome. All the risk transmission chains constructed are matched with the time evolution rules in the bidirectional correlation model of human factors and road conditions to verify whether the time interval of each link in the risk transmission chain meets the time threshold requirements in the time evolution rules, and risk transmission chains that do not meet the time threshold requirements are eliminated. For the verified risk transmission chain, supplement the missing feature change links in the chain so that the risk transmission chain fully reflects the transmission process from the initial triggering factor to the driving risk result; All the completed risk transmission chains were classified and organized into three groups according to the type of risk transmission initiation factors: human-induced triggering, road condition-induced triggering, and human-induced and road condition-induced triggering. The risk transmission chains in each group constitute the driving risk transmission subset under that type. All driving risk transmission subsets are integrated to form a driving risk transmission path set containing different types of risk transmission paths. Each driving risk transmission path corresponds to a unique path identifier in the set. The path identifier includes information on the type of initiating factor, the number of transmission links, and the type of risk outcome.

4. The method for dynamic early warning of driving risks based on the fusion of human factors and road conditions according to claim 1, characterized in that, The step of extracting path features for each driving risk transmission path in the set of driving risk transmission paths, and determining the risk impact range corresponding to each driving risk transmission path based on the extracted path features, includes: Select a driving risk transmission path from the set of driving risk transmission paths, extract all transmission links in the driving risk transmission path, and each transmission link corresponds to a feature change event. The feature change event includes the event type, the time of the event, and the human factors or road conditions that affect the event. The impact of each characteristic change event on vehicle driving parameters, including driving speed, driving direction, braking distance, and steering angle, is analyzed. By comparing the changes in vehicle driving parameters before and after the event, the independent impact value of each characteristic change event on each vehicle driving parameter is determined. The independent impact values ​​of each characteristic change event are normalized separately, and the normalized independent impact values ​​are combined to obtain the single, dimensionless parameter impact value of the characteristic change event. The parameter impact values ​​of all transmission links are accumulated in the order of transmission to obtain the total parameter impact value of the driving risk transmission path. During the accumulation process, each parameter impact value is weighted according to the position weight of the transmission link in the path. The weight coefficient value of the link at the front end of the path is less than the weight coefficient value of the link at the back end of the path. The transmission speed characteristics of the driving risk transmission path are extracted. The transmission speed characteristics are obtained by calculating the average time interval between adjacent transmission links. The smaller the average time interval value, the faster the transmission speed value corresponding to the transmission speed characteristics. Based on the total parameter impact value and transmission speed characteristics, a preset impact range mapping table is queried. The impact range mapping table stores the risk impact range descriptions corresponding to different total parameter impact value ranges and transmission speed ranges. The risk impact range descriptions include the affected vehicle system and the affected driving area. The risk outcome types in the driving risk transmission path are analyzed to determine the secondary risk types that the risk outcomes may cause. The risk impact range descriptions obtained from the query are supplemented with the secondary risk types to form risk impact range information. The secondary risk types include vehicle loss of control risk, rear-end collision risk, and lane departure risk. Repeat the above steps to extract path features and determine the risk impact range for each driving risk transmission path in the set of driving risk transmission paths, so that each driving risk path corresponds to unique risk impact range information; The path identifier of each driving risk transmission path is associated with and stored with the corresponding risk impact range information to form a path impact range association table. The path impact range association table also includes the total parameter impact value and transmission speed characteristics of each path.

5. The method for dynamic early warning of driving risks based on the fusion of human factors and road conditions according to claim 1, characterized in that, The process involves combining the risk impact range corresponding to each driving risk transmission path to generate a dynamic driving risk warning instruction containing a description of the risk transmission direction and risk impact range. This instruction is then sent to the driving control terminal, triggering the terminal to perform a warning notification operation, including: Extract the path identifier, risk impact range information, total parameter impact value, and transmission speed characteristics of all driving risk transmission paths from the path impact range association table; All driving risk transmission paths are prioritized based on the magnitude of the total parameter impact value. The larger the total parameter impact value, the higher the priority of the path. For paths with the same total parameter impact value, a secondary sort is performed based on the transmission speed characteristic. The faster the transmission speed, the higher the priority of the path. A preset number of driving risk transmission paths are selected as core early warning paths in descending order of priority. The preset number is determined based on the complexity of the current driving scenario. Extract the risk transmission path of each core early warning path, determine the characteristic change direction of each transmission link in the risk transmission path, and connect the characteristic change directions in the transmission order to obtain the risk transmission direction of the core early warning path. The risk transmission direction is represented by a sequence of direction vectors, and each direction vector corresponds to the characteristic change direction of a transmission link. Extract the affected vehicle systems and affected driving areas from the risk impact range information corresponding to each core early warning path, integrate the affected vehicle systems and affected driving areas into a risk impact range description, and use structured language to describe the risk impact range to determine the specific impact mode of each affected object; The path identifier, risk transmission direction, and risk impact range description of each core early warning path are integrated to form a single path early warning information. Each single path early warning information includes a path identifier field, a transmission direction field, and an impact range field. All core warning paths are summarized, and the current warning generation time and warning validity period information are added to generate a dynamic warning instruction for driving risks. The dynamic warning instruction for driving risks is written in a format that can be parsed by the driving control terminal, and includes an instruction header, instruction body, and instruction tail. The instruction header contains an instruction type identifier, the instruction body contains all single-path warning information, and the instruction tail contains relevant information. The generated dynamic driving risk warning command is sent to the driving control terminal. After receiving the command, the terminal parses the single-path warning information in the command body and determines the corresponding warning prompt method based on the risk transmission direction and risk impact range description. The determined warning prompt method is then executed, and the warning execution time, warning prompt method, and driver's response are recorded to form a warning execution log. The warning execution log is used for subsequent evaluation of the warning effect. Warning prompt methods include voice prompts, light prompts, and seat vibration prompts. Different risk impact range descriptions correspond to different combinations of warning prompts.

6. The method for dynamic early warning of driving risks based on the fusion of human factors and road conditions according to claim 2, characterized in that, The dynamic status data of drivers collected in chronological order is segmented, with each segment corresponding to a time window. The core features of the driver's dynamic status data within each time window are extracted to form driver status nodes. Similarly, the dynamic status data of security guards collected in the same time window is segmented, and the core features of the security guard's dynamic status data within each time window are extracted to form security guard status nodes, including: The duration of the time window is determined based on the vehicle's speed. The faster the speed, the shorter the time window; the slower the speed, the longer the time window. Starting from the start time of data collection, the collected dynamic status data of drivers is segmented according to the determined time window duration to obtain multiple segments of driver dynamic status data. Each segment of driver dynamic status data corresponds to a time window identifier. Statistical analysis was performed on the eye state data in each segment of driver dynamic state data. The average value of eyelid closure frequency and the coverage of eyeball rotation trajectory within the time window were calculated. The average value and coverage were used as the core features of eye state. The facial expression data in the segmented data of the driver's dynamic state were analyzed, and the maximum and minimum values ​​of the change in the corner of the mouth and the duration of the facial muscle contraction were extracted. The maximum, minimum and duration were taken as the core features of facial expression. The physiological dynamic data in the segmented data of the driver's dynamic state were statistically analyzed, and the fluctuation amplitude of heart rate change data, the average value of respiratory rate data, and the peak value of skin conductance response data were calculated. The fluctuation amplitude, average value, and peak value were used as the core physiological dynamic features. The core features of eye state, facial expression, and physiological dynamics are standardized respectively. The standardized core features of eye state, facial expression, and physiological dynamics are then integrated into a feature vector to form the driver state node corresponding to the time window. The driver state node includes the time window identifier and the corresponding feature vector. Using the same time window duration, the collected dynamic status data of the escorts are segmented to obtain multiple segments of escort dynamic status data. Each segment of escort dynamic status data corresponds to the same time window identifier as the driver's status node. The voice interaction data in each segment of the escort's dynamic status data is analyzed to extract the number of keywords and the number of tone changes in the voice communication content. The number of keywords and the number of tone changes are taken as the core features of voice interaction. The motion posture data in the segmented dynamic state data of the escort personnel were analyzed, and the average value of the limb movement amplitude and the angle between the direction of the movement and the direction of vehicle travel were calculated. The average value and the angle were used as the core features of the motion posture. The core features of voice interaction and the core features of action posture are standardized separately. The standardized core features of voice interaction and the core features of action posture are integrated into a feature vector to form the escort status node corresponding to the time window. The escort status node contains the same time window identifier and corresponding feature vector as the driver status node.

7. The method for dynamic early warning of driving risks based on the fusion of human factors and road conditions according to claim 1, characterized in that, The calculation of the interaction between driver's operational tendencies and road resistance characteristics, the interaction between driver fatigue characteristics and road visibility interference characteristics, and the interaction between escort personnel's prompting characteristics and road traffic conflict characteristics for the same person at road condition nodes include: Driver operation tendency features are extracted from the human state nodes of the same human-factor road condition node pair. These driver operation tendency features include steering operation frequency, braking operation frequency, and acceleration operation frequency. Road resistance features are extracted from the road condition nodes of the same node pair. These road resistance features include road gradient, road surface friction coefficient, and road smoothness. An interaction matrix is ​​established between driver operation tendency characteristics and road resistance characteristics. The rows of the interaction matrix represent the parameters of driver operation tendency characteristics, and the columns represent the parameters of road resistance characteristics. Each element in the matrix is ​​used to store the interaction influence value of the corresponding parameter pair. Based on historical driving data, the correlation coefficient between the standardized steering operation frequency and the standardized road slope is calculated as the interaction value; the correlation coefficient between the standardized braking operation frequency and the standardized road surface friction coefficient is calculated as the interaction value; the correlation coefficient between the standardized acceleration operation frequency and the standardized road smoothness is calculated as the interaction value. All interaction influence values ​​in the interaction matrix are normalized so that all values ​​are in the same numerical range. Then the trace of the interaction matrix is ​​calculated, and the trace value is used as the degree of interaction between driver operation tendency characteristics and road resistance characteristics. Driver fatigue features are extracted from the human state nodes of the same human-cause road condition node pair. Driver fatigue features include the proportion of eyelid closure time, heart rate variability coefficient, and reaction delay time. Road vision interference features are extracted from the road condition nodes of the same node pair. Road vision interference features include the amplitude of light intensity change, visibility caused by precipitation, and the range of obstruction by surrounding objects. Construct a correlation function between driver fatigue characteristics and road vision interference characteristics. The input of the correlation function is the parameters of driver fatigue characteristics and the parameters of road vision interference characteristics, and the output is the interaction influence value. The standardized eyelid closure duration percentage and the standardized light intensity variation amplitude are input into the first submodule of the correlation function to calculate their combined effect value. The combined effect value is the result of a function trained on historical data that outputs a dimensionless evaluation value. The standardized heart rate variability coefficient and the standardized visibility caused by precipitation are input into the second submodule of the correlation function to calculate their coupled effect value. The coupled effect value is the result of a function trained on historical data that outputs a dimensionless evaluation value. The standardized reaction delay time and the standardized occlusion range of surrounding objects are input into the third submodule of the correlation function to calculate their superimposed effect value. The superimposed effect value is the result of a function trained on historical data that outputs a dimensionless evaluation value. The output values ​​of the first submodule, the second submodule, and the third submodule are weighted and summed to obtain the degree of interaction between driver fatigue characteristics and road vision interference characteristics. Extract escort personnel prompt features from the human state nodes of the same human-cause road condition node pair. The escort personnel prompt features include prompt frequency, prompt accuracy, and prompt response time. Extract road traffic conflict features from the road condition nodes of the same node pair. The road traffic conflict features include the approach speed of adjacent vehicles, the number of lane occupancy conflicts, and the frequency of traffic signal changes. An impact assessment model is established for the characteristics of escort personnel prompts and road traffic conflicts. The impact assessment model includes an input layer, a processing layer, and an output layer. The input layer receives various parameters of the two types of features, the processing layer performs interactive calculations on the parameters, and the output layer outputs the degree of interactive impact. The input layer inputs the standardized alert frequency and the standardized approach speed of adjacent vehicles into the first computing unit of the processing layer to calculate the mitigation coefficient of the alert frequency on the risk of adjacent vehicle approach. The mitigation coefficient is the result of a function calculation based on historical data and outputting dimensionless coefficients. The input layer inputs the standardized prompt accuracy and the standardized number of lane occupancy conflicts into the second processing unit of the processing layer to calculate the lane occupancy conflict avoidance efficiency of the prompt accuracy. The avoidance efficiency is the result of a function calculation based on historical data training and outputting dimensionless coefficients. The input layer inputs the standardized prompt response time and the standardized traffic signal change frequency into the third processing unit of the processing layer to calculate the response time's effectiveness in responding to traffic signal change risks. The effectiveness is a function calculation result based on historical data training, outputting dimensionless coefficients. The processing layer comprehensively evaluates the output results of the first, second, and third operation units to determine the proportion of positive and negative impacts. The output layer calculates the degree of interaction between the escort officer prompting feature and the road traffic conflict feature based on the proportion of positive and negative impacts.

8. A dynamic early warning system for driving risks based on the fusion of human factors and road conditions, characterized in that, include: processor; A machine-readable storage medium for storing machine-executable instructions of the processor; The processor is configured to execute the dynamic warning method for driving risks based on the fusion of human factors and road conditions as described in any one of claims 1 to 7 by executing the machine-executable instructions.

9. A computer program product, characterized in that, The computer program product includes machine-executable instructions stored in a computer-readable storage medium. The processor of the dynamic warning system for driving risks based on the fusion of human factors and road conditions reads the machine-executable instructions from the computer-readable storage medium and executes the machine-executable instructions, causing the dynamic warning system for driving risks based on the fusion of human factors and road conditions to perform the dynamic warning method for driving risks based on the fusion of human factors and road conditions as described in any one of claims 1 to 7.

Images