Vehicle collision early warning method and system based on artificial intelligence

Abstract

The invention provides a vehicle collision early warning method and system based on artificial intelligence, and the method comprises the steps: firstly obtaining a target vehicle multi-source sensor data set which comprises vehicle operation state, environment perception and driving behavior data sequences, each sequence is composed of standardized monitoring data, then carrying out the dynamic spatial-temporal feature extraction of the multi-source sensor data set, and carrying out the early warning of vehicle collision. The method comprises the following steps: generating dynamic space-time coding characteristics containing characteristics such as a vehicle movement trend, calling a pre-trained collision risk prediction model to carry out multi-dimensional risk association analysis on the dynamic space-time coding characteristics to obtain a risk prediction result containing a longitudinal collision risk level and the like, and constructing a real-time early warning strategy set based on the risk prediction result. And finally, deploying the set to a vehicle-mounted decision-making system, establishing a control instruction mapping relation with a vehicle execution mechanism, triggering real-time risk intervention operation, and realizing effective collision early warning and risk intervention.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based vehicle collision warning method and system. Background Art

[0002] In the field of vehicle safety technology and auto insurance risk control, with the continuous growth of car ownership and the increasing complexity of the road traffic environment, vehicle collision accidents have become one of the main factors threatening road traffic safety and increasing the risk of auto insurance claims. Traditional vehicle collision warning systems mostly rely on single sensor data or simple threshold judgments, such as distance monitoring based only on radar or camera data, or setting fixed collision warning thresholds based on vehicle speed and distance. Although such methods can provide warning functions to a certain extent, they have significant limitations: on the one hand, a single data source is difficult to fully reflect the dynamic changes of the vehicle's driving status and the surrounding environment, resulting in limited warning accuracy; on the other hand, fixed threshold judgments cannot adapt to different driving scenarios and differences in driver behavior, and are prone to false alarms or missed alarms, affecting the effectiveness of the warning and user experience.

[0003] Furthermore, when dealing with vehicle collision risks, relevant technologies often lack comprehensive consideration of multi-dimensional information such as vehicle movement trends, environmental interactions, and abnormal driving behavior. For example, there is a lack of effective fusion analysis between vehicle operating status data (such as speed and acceleration) and environmental perception data (such as obstacle locations and road conditions), making it difficult to accurately predict potential collision risks; at the same time, the impact of driver behavior patterns (such as sudden acceleration, sudden braking, and frequent lane changes) on collision risks has not received sufficient attention, making it difficult for the early warning system to identify and intervene in high-risk driving behaviors in advance.

[0004] In terms of auto insurance risk control, existing technologies mainly rely on historical accident data and static risk assessment models, which make it difficult to reflect the dynamic risk changes during vehicle driving in real time. This lagging risk assessment method not only fails to effectively prevent accidents, but may also lead to problems such as unreasonable auto insurance pricing and increased compensation risks. Summary of the invention

[0005] In view of the above-mentioned problems, in combination with the first aspect of the present invention, an embodiment of the present invention provides a vehicle collision warning method based on artificial intelligence, the method comprising: Acquire a multi-source sensor data set of a target vehicle, wherein the multi-source sensor data set includes a vehicle operation status data sequence, an environment perception data sequence, and a driving behavior data sequence, each data sequence consisting of standardized monitoring data of multiple continuous time windows; Performing a dynamic spatiotemporal feature extraction operation on the multi-source sensor data set to generate dynamic spatiotemporal coding features for each time window, wherein the dynamic spatiotemporal coding features include vehicle motion trend features, environmental interaction features, and behavior abnormality features; Calling a pre-trained collision risk prediction model to perform multi-dimensional risk association analysis on the dynamic spatiotemporal coding features to generate a risk prediction result for the current time window, wherein the risk prediction result includes a longitudinal collision risk level, a lateral deviation risk level, and an emergency braking trigger probability; Building a real-time warning strategy set based on the risk prediction result, the real-time warning strategy set includes warning signal generation rules and vehicle control instruction sequences corresponding to different risk levels; The real-time warning strategy set is deployed to the vehicle-mounted decision-making system, and a control instruction mapping relationship with the vehicle actuator is established to trigger real-time risk intervention operations.

[0006] On the other hand, an embodiment of the present invention also provides an artificial intelligence-based vehicle collision warning system, including a processor and a machine-readable storage medium, wherein the machine-readable storage medium is connected to the processor, the machine-readable storage medium is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the machine-readable storage medium to implement the above method.

[0007] Based on the above aspects, the embodiment of the present invention integrates multi-source sensor data sets such as vehicle operation status, environmental perception and driving behavior. On this basis, the dynamic spatiotemporal feature extraction operation not only captures key information such as vehicle movement trends and environmental interactions, but also introduces behavioral abnormality characteristics, effectively identifies potential human driving risks, and enhances the foresight and accuracy of risk identification. Furthermore, the pre-trained collision risk prediction model converts dynamic spatiotemporal coding features into risk prediction results through multi-dimensional risk association analysis, including longitudinal collision risk level, lateral deviation risk level and emergency braking trigger probability, realizes quantitative assessment and graded warning of collision risks, and provides scientific and quantitative decision-making basis for vehicle insurance risk control. The real-time warning strategy set constructed based on the risk prediction results dynamically generates warning signals and control instructions according to the risk level, ensuring the timeliness of warnings and the effectiveness of intervention, and effectively reducing the probability of accidents and the degree of losses. Finally, the real-time warning strategy set is deployed to the on-board decision-making system, and a control command mapping relationship with the vehicle's actuator is established, realizing the connection from risk warning to active intervention, forming an active protection network for safe vehicle driving, which not only improves the safety of vehicle driving, but also reduces the risk of auto insurance claims, and realizes intelligent and precise risk control measures. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1It is a schematic diagram of the execution flow of the vehicle collision warning method based on artificial intelligence provided by an embodiment of the present invention.

[0009] Figure 2 Schematic diagram of exemplary hardware and software components of an artificial intelligence-based vehicle collision warning system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0010] The present invention will be described in detail below with reference to the accompanying drawings. Figure 1 The present invention is a flowchart of an artificial intelligence-based vehicle collision warning method provided by an embodiment of the present invention. The artificial intelligence-based vehicle collision warning method is introduced in detail below.

[0011] Step S110, obtaining a multi-source sensor data set of the target vehicle, wherein the multi-source sensor data set includes a vehicle operating status data sequence, an environment perception data sequence, and a driving behavior data sequence, and each data sequence is composed of standardized monitoring data of multiple continuous time windows.

[0012] In this embodiment, in an intelligent traffic environment, a target vehicle equipped with various sensors and intelligent systems is driving on a road. The road conditions are relatively complex, with moderate traffic volume, various obstacles and different road signs around. The target vehicle is designed to monitor and analyze various data in real time through the intelligent system it carries, so as to prevent potential collision risks and ensure driving safety.

[0013] For example, the target vehicle is equipped with multiple types of sensors to collect data from different aspects. In terms of the vehicle operation status data sequence, the longitudinal acceleration sensor continuously collects longitudinal acceleration data and records the value every 0.1 seconds. For example, the longitudinal acceleration data sequence recorded in a certain period of time is [1.2, 1.5, 1.3, 1.4, 1.6] (unit: m / s²); the lateral angular velocity sensor synchronously collects lateral angular velocity data, also at intervals of 0.1 seconds, and obtains a data sequence such as [0.2, 0.3, 0.25, 0.32, 0.28] (unit: rad / s); the brake pressure sensor records the brake pressure data, and obtains a brake pressure data sequence such as [20, 22, 21, 23, 22] (unit: kPa). After being collected, the above data can be processed according to the set standardization rules. For example, the longitudinal acceleration data can be mapped to the interval [0, 1] by subtracting the minimum value in the data sequence from each data and then dividing it by the difference between the maximum and minimum values. Assuming that the minimum longitudinal acceleration is 1.2 and the maximum is 1.6, the first data 1.2 is standardized to (1.2-1.2) / (1.6-1.2)=0, and 1.6 is standardized to (1.6-1.2) / (1.6-1.2)=1, and so on to standardize the entire sequence.

[0014] In terms of acquiring environmental perception data sequences, sensors such as millimeter-wave radars and cameras installed around the vehicle acquire the relative position data of obstacles around the target vehicle in real time. The millimeter-wave radar measures the relative position of the obstacle and the vehicle every 0.2 seconds to generate a relative position data sequence. Assume that the relative position data of an obstacle at several time points is [(20, 5), (18, 6), (16, 7), (14, 8)] (unit: meter, representing the lateral and longitudinal distances respectively); at the same time, the lane recognition camera monitors the lane and collects the lane curvature data sequence, such as [0.01, 0.012, 0.011, 0.013, 0.012]; the ambient light sensor collects the ambient light intensity data sequence at intervals of 1 second, such as [500, 520, 510, 530, 525] (unit: lux). The above data are also standardized. For example, for relative position data, the lateral and longitudinal distances are mapped to the interval [0, 1] respectively, and processed by a method similar to the above-mentioned longitudinal acceleration data standardization method.

[0015] The driving behavior data sequence is collected through the steering wheel angle sensor, pedal position sensor and driver sight monitoring device. The steering wheel angle sensor records the steering wheel angle change in real time and generates the angle change data sequence at intervals of 0.1 seconds, such as [10, 12, 11, 13, 12] (unit: degree); the pedal position sensor monitors the pedal position switching, generates the pedal position switching data sequence, and records the time of each pedal switching, such as [0.5, 1.2, 1.8, 2.5, 3.0] (unit: second, indicating the time interval from the last switch to this switch); the driver sight monitoring device tracks the driver's sight direction through a camera, generates the sight direction data sequence, and records the duration of each sight deviation from the front of the vehicle and the lateral position offset of the vehicle, such as [(3, 0.1), (4, 0.12), (3.5, 0.11), (4.5, 0.13), (4, 0.12)] (unit: second and meter). The above data are also standardized. For example, the angle change data is mapped to the interval [0, 1]. In the above manner, a multi-source sensor data set consisting of a vehicle operation status data sequence, an environmental perception data sequence, and a driving behavior data sequence is finally obtained. Each data sequence consists of standardized monitoring data of multiple continuous time windows.

[0016] Step S120, performing a dynamic spatiotemporal feature extraction operation on the multi-source sensor data set to generate dynamic spatiotemporal coding features for each time window, wherein the dynamic spatiotemporal coding features include vehicle motion trend features, environmental interaction features, and behavior abnormality features.

[0017] In this embodiment, for the vehicle running status data sequence, the longitudinal acceleration data sequence, the lateral angular velocity data sequence and the brake pressure data sequence are first collected. For the longitudinal acceleration data sequence, a first time convolution network is constructed. The first time convolution network includes alternating dilated convolution layers and normalization layers. The dilation coefficient of the dilated convolution layer is set to 2 to capture multi-scale acceleration pattern characteristics. For example, by performing a convolution operation on the longitudinal acceleration data through the dilated convolution layer, some periodic or trend characteristics in the data can be found. Assume that after processing by the dilated convolution layer, some characteristic values ​​such as [0.3, 0.4, 0.35, 0.42, 0.38] are obtained, and then the above characteristic values ​​are mapped to a suitable range, such as the [0, 1] interval, through the normalization layer to obtain the acceleration time series characteristics.

[0018] For the lateral angular velocity data sequence, a second temporal convolutional network is constructed, which includes a bidirectional convolutional layer and a gated activation unit. The bidirectional convolutional layer can convolve the data from both the forward and reverse directions to better extract the temporal dependency features of the steering operation. For example, by processing the lateral angular velocity data sequence through the bidirectional convolutional layer, the sequence and change pattern of the steering operation can be discovered. The gated activation unit screens and activates the data according to its importance, and obtains the steering temporal features after processing.

[0019] For the brake pressure data sequence, a third temporal convolutional network is constructed, which includes a residual convolution module and an attention pooling layer. The residual convolution module can retain some key information in the data to prevent information loss during the convolution process. The attention pooling layer focuses on the important features of the braking behavior, identifies the sudden features of the braking behavior, and generates braking time series features.

[0020] The acceleration time series characteristics are differentially calculated to obtain the acceleration change rate characteristics. For example, if the acceleration time series characteristics are [0.3, 0.4, 0.35, 0.42, 0.38], then the acceleration change rate characteristics are calculated as [(0.4-0.3), (0.35-0.4), (0.42-0.35), (0.38-0.42)] = [0.1, -0.05, 0.07, -0.04]. The steering angular velocity fluctuation feature is obtained by performing sliding window variance calculation on the steering timing feature. Assuming the sliding window size is 3 and the steering timing feature is [0.2, 0.3, 0.25, 0.32, 0.28], the variance of the first sliding window [0.2, 0.3, 0.25] is calculated as [(0.2-(0.2+0.3+0.25) / 3)²+(0.3-(0.2+0.3+0.25) / 3)²+(0.25-(0.2+0.3+0.25) / 3)²] / 3. Similarly, the variance of the entire sequence is calculated to obtain the steering angular velocity fluctuation feature. The braking pressure gradient feature is obtained by performing gradient analysis on the braking timing feature. Thus, the vehicle motion trend feature is obtained by splicing the above features.

[0021] For the environmental perception data sequence, the relative position data sequence of obstacles around the target vehicle, the lane curvature data sequence, and the ambient light intensity data sequence are obtained. The obstacle motion trajectory prediction model is constructed, and the motion state vector of the obstacle is initialized. It is assumed that it contains 20 lateral position observation values ​​(unit: meter), 10 longitudinal speed observation values ​​(unit: m / s), and 0.5 acceleration estimation values ​​(unit: m / s²). The state transfer matrix and the observation noise covariance matrix are generated based on the vehicle coordinate system. The state transfer matrix is ​​determined according to the vehicle kinematic principle. The motion state vector is predicted iteratively by the state transfer matrix. Assuming that the time step is 0.2 seconds, the predicted state vector and the corresponding predicted covariance matrix are output after one iterative prediction. The residual of the predicted state vector and the actual observation value of the current time window is calculated. For example, the predicted lateral position is 18 meters, the actual observation value is 17 meters, and the residual is 1 meter. The Mahalanobis distance of the residual vector is calculated in combination with the predicted covariance matrix. When the Mahalanobis distance exceeds the dynamically adjusted threshold, assuming the threshold is 0.8 and the actual calculated Mahalanobis distance is 0.9, the noise parameters of the observation noise covariance matrix are corrected by a scaling factor of 1.2 to generate an updated observation noise covariance matrix. The updated observation noise covariance matrix is ​​used to re-execute the measurement update step of the Kalman filter, and the corrected prediction state vector and lateral displacement prediction error characteristics are output. The corrected prediction state vector is input into the trajectory pattern matching network, and a convolution similarity comparison is performed with the typical motion pattern in the historical trajectory database to generate the longitudinal velocity covariance feature. The lateral displacement prediction error feature and the longitudinal velocity covariance feature are integrated to generate the obstacle relative motion trajectory feature.

[0022] The pre-trained lane feature encoder is called to extract multi-scale features from the lane curvature data sequence, generating lane curvature matching features, including the dynamic matching coefficient between the current lane curvature and the vehicle steering angle. The ambient light intensity data sequence is statistically processed using a sliding window. Assuming the sliding window size is 3, the visibility influence coefficient feature is generated, including the associated weight parameter between the light intensity change rate and the obstacle recognition confidence. Thus, the above features are fused by spatial position encoding to obtain the environmental interaction feature.

[0023] For the driving behavior data sequence, the steering wheel angle change data sequence, the pedal position switching data sequence and the driver's line of sight direction data sequence are collected. The angle change data sequence is pre-processed by denoising, and the high-frequency noise component is removed by wavelet threshold denoising algorithm to obtain the denoised angle signal. The denoised angle signal is short-time Fourier transformed to generate a time-frequency energy distribution matrix. Within the set frequency band of the time-frequency energy distribution matrix, assuming that the frequency band is [10-20Hz], the energy peak feature is extracted, and the ratio parameter of the energy in the set frequency band to the total energy is calculated based on the energy peak feature. The ratio parameter of 5 consecutive time windows is processed by sliding average to generate the steering wheel operation frequency feature. The pedal position switching data sequence is analyzed for event intervals to generate the pedal switching interval feature, including the time interval distribution feature of the alternating operation of the accelerator pedal and the brake pedal. The line of sight direction data sequence is subjected to continuous deviation detection to generate the line of sight deviation duration feature, including the correlation coefficient between the duration of a single line of sight deviation event and the lateral position offset of the vehicle. Therefore, the behavior abnormality feature is obtained by standardizing and splicing the above features.

[0024] Finally, the vehicle motion trend features, environmental interaction features and behavior abnormality features are temporally aligned and fused to generate dynamic spatiotemporal coding features.

[0025] Step S130, calling a pre-trained collision risk prediction model to perform multi-dimensional risk association analysis on the dynamic spatiotemporal coding features to generate a risk prediction result for the current time window, wherein the risk prediction result includes a longitudinal collision risk level, a lateral deviation risk level, and an emergency braking trigger probability.

[0026] In this embodiment, the dynamic spatiotemporal coding features can be input into the spatiotemporal attention module of the collision risk prediction model, and the dynamic spatiotemporal coding features of the current time window and the dynamic spatiotemporal coding features of the historical time window are input into the linear mapping layer of the spatiotemporal attention module to generate the current query feature vector, the historical key feature vector and the historical value feature vector, respectively, wherein the historical key feature vector and the historical value feature vector are arranged in the order of the time window. For example, the current query feature vector is [0.3, 0.4, 0.5], and the historical key feature vectors in the past three time windows are [0.2, 0.3, 0.4], [0.3, 0.4, 0.5], [0.4, 0.5, 0.6], and the historical value feature vectors are [0.4, 0.5, 0.6], [0.5, 0.6, 0.7], [0.6, 0.7, 0.8].

[0027] The initial time window association sequence is generated by performing a dot product operation on the current query feature vector and the key feature vector of each historical time window. For example, the dot product with the first historical key feature vector is [0.3×0.2+0.4×0.3+0.5×0.4]=0.38, and the initial time window association sequence is obtained by analogy. The initial time window association sequence is subjected to an exponential scaling operation with a learnable scaling factor of 1.5 to generate the attention energy distribution before normalization. Softmax normalization calculation is performed on the attention energy distribution along the time window dimension to generate the weight allocation coefficient for each historical time window. The weight allocation coefficient is weightedly superimposed on the historical value feature vector in the order of the time window to generate the feature weight distribution of different time windows.

[0028] Based on the feature weight distribution, the features of the historical time window are dynamically weighted and fused to generate an enhanced spatiotemporal feature representation. The enhanced spatiotemporal feature representation is input into the risk classifier of the collision risk prediction model, which includes a longitudinal risk branch, a lateral risk branch, and an emergency braking branch that are set in parallel. The longitudinal risk branch analyzes the input features and outputs the longitudinal collision risk level, for example, it is divided into three levels: low, medium, and high; the lateral risk branch outputs the lateral deviation risk level, which is also divided into different levels; the emergency braking branch outputs the emergency braking trigger probability, for example, the output probability is 0.2. The output results of the longitudinal risk branch, the lateral risk branch, and the emergency braking branch that are set in parallel are fused at the decision level to generate a risk prediction result.

[0029] Step S140: constructing a real-time warning strategy set based on the risk prediction result, wherein the real-time warning strategy set includes warning signal generation rules and vehicle control instruction sequences corresponding to different risk levels.

[0030] For example, assuming that the longitudinal collision risk level is high, the warning signal generation rule is to issue a voice prompt through the in-vehicle voice system that "there is a high collision risk ahead, please take immediate action", and the warning light on the dashboard flashes red; the vehicle control command sequence is to automatically start the emergency braking system and adjust the brake pressure to the maximum allowable value, such as 500kPa. When the longitudinal collision risk level is medium, the warning signal generation rule is a voice prompt "there is a medium collision risk ahead, please pay attention", and the warning light flashes yellow; the vehicle control command sequence is to appropriately reduce the speed, such as reducing the speed by 20%. When the longitudinal collision risk level is low, the warning signal generation rule is a voice prompt "there is a low collision risk ahead, stay alert", and the warning light flashes green; the vehicle control command sequence is to maintain the current speed.

[0031] For the lateral deviation risk level, when it is a high level, the warning signal generation rule is to remind the driver by vibrating the seat and issuing a voice prompt of "the vehicle has a serious lateral deviation risk"; the vehicle control command sequence is to automatically adjust the steering system and adjust the steering angle to an appropriate value to correct the lateral deviation. When it is a medium level, the warning signal generation rule is to slightly vibrate the steering wheel and give a voice prompt of "the vehicle has a medium lateral deviation risk"; the vehicle control command sequence is to fine-tune the steering angle appropriately. When it is a low level, the warning signal generation rule is to display a prompt message on the dashboard that "the vehicle has a low lateral deviation risk"; the vehicle control command sequence is not to actively intervene, but to continuously monitor.

[0032] For the probability of emergency braking triggering, when the probability exceeds 0.5, the warning signal generation rule is to issue a sharp alarm and the warning light flashes red quickly; the vehicle control command sequence is to immediately start the emergency braking system. When the probability is between 0.3-0.5, the warning signal generation rule is a voice prompt "The possibility of emergency braking is high, please prepare", and the warning light flashes orange; the vehicle control command sequence is to pre-charge the braking system to increase the braking response speed. When the probability is lower than 0.3, the warning signal generation rule is not to issue a special warning; the vehicle control command sequence is normal driving monitoring. Through the above rules, a real-time warning strategy set is constructed.

[0033] Step S150, deploying the real-time warning strategy set to the vehicle-mounted decision-making system, and establishing a control instruction mapping relationship with the vehicle actuator to trigger a real-time risk intervention operation.

[0034] In this embodiment, the constructed real-time warning strategy set can be written into the storage module of the vehicle-mounted decision-making system, and the warning signal generation rules and vehicle control instruction sequence and other information can be accurately transmitted to the corresponding module through the vehicle network. For example, the warning signal generation rules and vehicle control instruction sequences when the longitudinal collision risk level is high are stored in a special risk processing module. A control instruction mapping relationship with the vehicle actuator is established. For the braking system, when the vehicle control instruction for emergency braking is received, the instruction is sent to the braking execution unit through the controller area network (CAN) bus, and the braking execution unit adjusts the braking pressure to the set value according to the instruction. For the steering system, when the instruction to adjust the steering angle is received, the instruction is transmitted to the steering motor through the electronic control unit (ECU), and the steering motor adjusts the steering angle according to the instruction. In this way, when the risk prediction result triggers the corresponding warning level during the operation of the vehicle, the vehicle-mounted decision-making system can quickly trigger the real-time risk intervention operation according to the real-time warning strategy set through the control instruction mapping relationship with the vehicle actuator to ensure the safety of vehicle driving.

[0035] Based on the above steps, the embodiment of the present invention integrates multi-source sensor data sets such as vehicle operating status, environmental perception and driving behavior. On this basis, the dynamic spatiotemporal feature extraction operation not only captures key information such as vehicle movement trends and environmental interactions, but also introduces behavioral abnormality characteristics, effectively identifies potential human driving risks, and enhances the foresight and accuracy of risk identification. Furthermore, the pre-trained collision risk prediction model converts dynamic spatiotemporal coding features into risk prediction results through multi-dimensional risk association analysis, including longitudinal collision risk level, lateral deviation risk level and emergency braking trigger probability, realizes quantitative assessment and graded warning of collision risks, and provides scientific and quantitative decision-making basis for vehicle insurance risk control. The real-time warning strategy set constructed based on the risk prediction results dynamically generates warning signals and control instructions according to the risk level, ensuring the timeliness of warnings and the effectiveness of intervention, and effectively reducing the probability of accidents and the degree of losses. Finally, the real-time warning strategy set is deployed to the on-board decision-making system, and a control command mapping relationship with the vehicle's actuator is established, realizing the connection from risk warning to active intervention, forming an active protection network for safe vehicle driving, which not only improves the safety of vehicle driving, but also reduces the risk of auto insurance claims, and realizes intelligent and precise risk control measures.

[0036] In a possible implementation, step S120 includes: Step S121, performing motion feature extraction processing on the vehicle running status data sequence to obtain vehicle motion trend features, wherein the vehicle motion trend features include acceleration change rate features, steering angular velocity features and brake pressure gradient features.

[0037] In a possible implementation, step S121 includes: Step S1211, collecting the longitudinal acceleration data sequence, lateral angular velocity data sequence and brake pressure data sequence of the target vehicle.

[0038] For example, the high-precision sensor installed on the target vehicle continuously collects longitudinal acceleration data sequence, lateral angular velocity data sequence and brake pressure data sequence. During a certain period of driving time, the longitudinal acceleration sensor records the longitudinal acceleration value every 0.1 seconds to form a longitudinal acceleration data sequence, for example, the recorded data is [1.0, 1.2, 1.4, 1.6, 1.8] (unit: m / s²); the lateral angular velocity sensor synchronously collects lateral angular velocity data every 0.1 seconds to obtain a lateral angular velocity data sequence, such as [0.1, 0.2, 0.3, 0.4, 0.5] (unit: rad / s); the brake pressure sensor also records the brake pressure data at the same time interval to generate a brake pressure data sequence, such as [15, 18, 21, 24, 27] (unit: kPa).

[0039] Step S1212, constructing multiple parallel time convolution networks to process the longitudinal acceleration data sequence, the lateral angular velocity data sequence and the brake pressure data sequence respectively, and generating acceleration timing features, steering timing features and braking timing features.

[0040] In one possible implementation, step S1212 includes: Step S1212-1, constructing a first time convolution network for the longitudinal acceleration data sequence, wherein the first time convolution network comprises alternating dilated convolution layers and normalization layers for capturing multi-scale acceleration pattern features.

[0041] For example, the expansion coefficient of the dilated convolution layer is set to 3, and its function is to capture the multi-scale acceleration pattern characteristics. When the longitudinal acceleration data sequence is convolved by the dilated convolution layer, the data can be processed with an interval of 3. For example, for the first data 1.0 in the longitudinal acceleration data sequence, the dilated convolution layer will combine the subsequent data with an interval of 3 for feature extraction to find the potential periodic or trend characteristics in the data. After processing by the dilated convolution layer, a series of eigenvalues ​​are obtained, such as [0.2, 0.3, 0.4, 0.5, 0.6], and then the above eigenvalues ​​are mapped to the [0, 1] interval after passing through the normalization layer. The specific normalization process is to first find the minimum value 0.2 and the maximum value 0.6 in this set of eigenvalues, and for each eigenvalue, calculate it by the formula (eigenvalue-minimum value) / (maximum value-minimum value). For example, the first eigenvalue is 0.2, which is calculated as (0.2-0.2) / (0.6-0.2)=0; for the eigenvalue 0.6, it is calculated as (0.6-0.2) / (0.6-0.2)=1, and so on. The entire sequence is standardized and the acceleration time series characteristics are finally obtained.

[0042] Step S1212-2, constructing a second time convolution network for the lateral angular velocity data sequence, wherein the second time convolution network comprises a bidirectional convolution layer and a gated activation unit, and is used to extract the timing-dependent features of the steering operation.

[0043] For example, the bidirectional convolution layer can convolve the lateral angular velocity data from both the forward and reverse directions, which can better extract the timing-dependent features of the steering operation. For example, when processing the lateral angular velocity data sequence [0.1, 0.2, 0.3, 0.4, 0.5], the bidirectional convolution layer can find the trend of the lateral angular velocity gradually increasing over time from the forward convolution, and the stability of this trend can be verified from the reverse convolution. The gated activation unit screens and activates the convolution results according to the importance of the data. After this series of processing, the steering timing features are generated.

[0044] Step S1212-3, constructing a third temporal convolutional network for the brake pressure data sequence, wherein the third temporal convolutional network comprises a residual convolution module and an attention pooling layer, and is used to identify the burst characteristics of the braking behavior.

[0045] For example, the residual convolution module is used to retain key information in the brake pressure data to prevent important information from being lost during the convolution process. The attention pooling layer focuses on the key features of the braking behavior to identify the sudden features of the braking behavior. For example, when the brake pressure data sequence suddenly increases, the attention pooling layer can highlight this feature. After being processed by the network, the braking time series features are obtained.

[0046] Step S1212-4, performing feature dimensionality reduction processing on the outputs of the first time convolution network, the second time convolution network, and the third time convolution network, respectively, to obtain the acceleration timing features, the steering timing features, and the braking timing features.

[0047] Step S1213, performing differential calculation on the acceleration time series characteristics to obtain acceleration change rate characteristics, performing sliding window variance calculation on the steering time series characteristics to obtain steering angular velocity fluctuation characteristics, and performing gradient analysis on the braking time series characteristics to obtain braking pressure gradient characteristics.

[0048] Assuming that the acceleration time series feature is [0.2, 0.3, 0.4, 0.5, 0.6], the difference calculation is to subtract the previous feature value from the next feature value. The first acceleration change rate feature value is 0.3-0.2=0.1, the second is 0.4-0.3=0.1, the third is 0.5-0.4=0.1, and the fourth is 0.6-0.5=0.1, so the acceleration change rate feature is [0.1, 0.1, 0.1, 0.1].

[0049] Further assume that the sliding window size is set to 3. For the steering time series feature [0.2, 0.3, 0.4, 0.5, 0.6], the first sliding window contains data 0.2, 0.3, and 0.4. First, calculate the average of these three data, that is, (0.2+0.3+0.4) / 3=0.3. Then calculate the variance. The calculation process of the variance is the sum of the squares of the difference between each data and the average value and then divided by the number of data. That is, [(0.2-0.3)²+(0.3-0.3)²+(0.4-0.3)²] / 3=(0.01+0+0.01) / 3≈0.0067. In the same way, the subsequent sliding window data are calculated in turn to obtain the steering angular velocity fluctuation characteristics.

[0050] Furthermore, gradient analysis can be understood as calculating the rate of change between two adjacent braking sequence characteristic values. Assuming that the braking sequence characteristic is [0.3, 0.5, 0.7, 0.9, 1.1], the first braking pressure gradient characteristic value is (0.5-0.3) / (0.1)=2 (0.1 here is the time interval), the second is (0.7-0.5) / (0.1)=2, the third is (0.9-0.7) / (0.1)=2, and the fourth is (1.1-0.9) / (0.1)=2, so the braking pressure gradient characteristic is [2, 2, 2, 2].

[0051] Step S1214, combining the acceleration change rate feature, the steering angular velocity fluctuation feature and the brake pressure gradient feature to obtain the vehicle motion trend feature.

[0052] For example, the acceleration change rate characteristic is [0.1, 0.1, 0.1, 0.1], the steering angular velocity fluctuation characteristic is [0.0067, 0.008, 0.009, 0.01], and the brake pressure gradient characteristic is [2, 2, 2, 2]. After splicing, the vehicle motion trend characteristic is [0.1, 0.1, 0.1, 0.1, 0.0067, 0.008, 0.009, 0.01, 2, 2, 2, 2]. The vehicle motion trend characteristic comprehensively reflects the motion trend characteristics of the vehicle during operation.

[0053] Step S122, performing spatial correlation analysis processing on the environmental perception data sequence to extract environmental interaction features, wherein the environmental interaction features include obstacle relative motion trajectory features, lane line curvature matching degree features, and visibility influence coefficient features.

[0054] In a possible implementation, step S122 includes: Step S1221, obtaining a relative position data sequence of obstacles around the target vehicle, a lane line curvature data sequence, and an ambient light intensity data sequence.

[0055] In this embodiment, the sensors around the target vehicle continuously obtain the relative position data sequence of the surrounding obstacles, the lane curvature data sequence and the ambient light intensity data sequence. The millimeter wave radar measures the relative position of the obstacle and the vehicle every 0.2 seconds to generate a relative position data sequence. For example, the relative position data of an obstacle at several time points are [(15, 4), (13, 5), (11, 6), (9, 7)] (unit: meter, representing the lateral and longitudinal distances respectively); the lane recognition camera monitors the lane and collects the lane curvature data sequence, such as [0.008, 0.01, 0.012, 0.014, 0.016]; the ambient light sensor collects the ambient light intensity data sequence at intervals of 1 second, such as [450, 480, 510, 540, 570] (unit: lux).

[0056] Step S1222, constructing an obstacle motion trajectory prediction model, performing motion state estimation on the relative position data sequence based on a Kalman filter algorithm, and generating obstacle relative motion trajectory features, wherein the obstacle relative motion trajectory features include lateral displacement prediction error features and longitudinal velocity covariance features.

[0057] In a possible implementation, step S1222 includes: Step S1222-1, initialize the motion state vector of the obstacle, wherein the motion state vector includes a lateral position observation value, a longitudinal velocity observation value, and an acceleration estimation value.

[0058] In a given intelligent traffic scenario, the target vehicle is continuously moving, and the surrounding environment is complex and dynamically changing, so accurate prediction of the obstacle's trajectory is crucial. At this moment, the millimeter-wave radar continuously measures the relative position of obstacles around the target vehicle at intervals of 0.2 seconds, generating a relative position data sequence.

[0059] In this step, it is assumed that at a certain moment, the millimeter wave radar measurement shows that the lateral position observation value of an obstacle relative to the target vehicle is 25 meters, the longitudinal velocity observation value is 12 meters per second, and the acceleration estimation value is set to 0.4 meters per second squared. Thus, by combining the above values ​​together, the obstacle motion state vector containing the lateral position observation value, the longitudinal velocity observation value and the acceleration estimation value is formed, that is, [25, 12, 0.4].

[0060] Step S1222-2, generating a state transfer matrix and an observation noise covariance matrix based on the vehicle coordinate system, performing time step iterative prediction on the motion state vector through the state transfer matrix, and outputting the predicted state vector and the corresponding predicted covariance matrix.

[0061] In this embodiment, the state transfer matrix is ​​generated based on the vehicle kinematics principle, which describes the law of change of the state of the obstacle within a time step. The time step is set to 0.2 seconds. Within this time interval, according to the kinematic formula, the lateral position change of the obstacle is related to the longitudinal velocity and acceleration, and the change of the longitudinal velocity depends on the acceleration. After a series of calculations based on the kinematic principle, the state transfer matrix is ​​determined. For example, the calculation of the lateral position takes into account the effect of the current longitudinal velocity and acceleration within 0.2 seconds, and the calculation of the longitudinal velocity is simply based on the cumulative effect of the acceleration within 0.2 seconds. The observation noise covariance matrix comprehensively considers the noise factors existing in the millimeter wave radar measurement process. Through statistical analysis of a large amount of historical measurement data, the values ​​of each element of the observation noise covariance matrix are determined to reflect the distribution of measurement errors.

[0062] In detail, based on the current motion state vector [25, 12, 0.4], the calculation is performed according to the state transfer matrix. For the lateral position, the calculation is the current lateral position plus the longitudinal velocity multiplied by the time step plus 0.5 times the acceleration multiplied by the square of the time step, that is, 25+12×0.2+0.5×0.4×(0.2)²=27.408 meters; the longitudinal velocity is the current longitudinal velocity plus the acceleration multiplied by the time step, that is, 12+0.4×0.2=12.08 meters per second; the acceleration remains unchanged at 0.4 meters per second squared. In this way, the predicted state vector [27.408, 12.08, 0.4] and the corresponding predicted covariance matrix are obtained. The predicted covariance matrix reflects the degree of uncertainty of this prediction.

[0063] Step S1222-3, performing residual calculation on the predicted state vector and the actual observation value of the current time window, and calculating the Mahalanobis distance of the residual vector in combination with the predicted covariance matrix.

[0064] Assume that the lateral position of the obstacle actually observed by the millimeter-wave radar in the current time window is 27 meters, and the longitudinal speed is 12.1 meters per second. The lateral position residual is the predicted lateral position minus the actual observed lateral position, that is, 27.408-27=0.408 meters; the longitudinal speed residual is the predicted longitudinal speed minus the actual observed longitudinal speed, that is, 12.08-12.1=-0.02 meters per second. This forms the residual vector [0.408, -0.02]. The Mahalanobis distance of the residual vector is calculated in combination with the predicted covariance matrix. The calculation of the Mahalanobis distance needs to consider the relationship between the residual vector and the predicted covariance matrix. The specific calculation process is to perform matrix multiplication on the residual vector and the inverse matrix of the predicted covariance matrix, then perform dot product operation on the result and the residual vector, and finally take the square root of the dot product result. After a series of matrix operations and numerical calculations, the value of the Mahalanobis distance is obtained.

[0065] Step S1222-4: when the Mahalanobis distance exceeds the dynamic adjustment threshold, the noise parameters of the observation noise covariance matrix are corrected by a scaling factor to generate an updated observation noise covariance matrix.

[0066] Assume that the dynamic adjustment threshold is set to 0.3, and the actual calculated Mahalanobis distance is 0.35, which exceeds the threshold. At this time, the noise parameters of the observation noise covariance matrix are corrected by the scaling factor 1.3. Specifically, each element in the observation noise covariance matrix is ​​multiplied by the scaling factor 1.3 to generate an updated observation noise covariance matrix.

[0067] Step S1222-5, use the updated observation noise covariance matrix to re-execute the measurement update step of the Kalman filter, and output the corrected prediction state vector and lateral displacement prediction error characteristics.

[0068] For example, in this process, the updated observation noise covariance matrix is ​​used, combined with the actual observation value and the previous prediction state vector, and a series of set calculation methods are used to readjust the prediction state vector. After calculation, the corrected prediction state vector is obtained. At the same time, the lateral displacement prediction error characteristic is calculated, that is, the difference between the corrected predicted lateral position and the actual observed lateral position. It is assumed that the lateral displacement prediction error characteristic is 0.3 meters.

[0069] Step S1222-6: input the corrected predicted state vector into the trajectory pattern matching network, perform convolution similarity comparison with the typical motion patterns in the historical trajectory database, and generate longitudinal velocity covariance features.

[0070] The trajectory pattern matching network stores a historical trajectory database, which contains various typical motion patterns. The modified predicted state vector is compared with the typical motion patterns in the historical trajectory database for convolution similarity. By performing convolution operations on different typical motion patterns and the modified predicted state vector, the similarity scores between them are calculated. For example, for each typical motion pattern, it is matched and calculated with the modified predicted state vector in the time and space dimensions to obtain a similarity value. By comparing all similarity values, the most matching typical motion pattern is found, and the longitudinal velocity covariance feature is generated based on the matching situation. Assume that the generated longitudinal velocity covariance feature is 0.5.

[0071] Step S1222-7, integrating the lateral displacement prediction error feature and the longitudinal velocity covariance feature to generate the obstacle relative motion trajectory feature.

[0072] For example, the lateral displacement prediction error feature of 0.3 meters and the longitudinal velocity covariance feature of 0.5 are combined to form the obstacle relative motion trajectory feature [0.3, 0.5]. This obstacle relative motion trajectory feature comprehensively reflects the motion trajectory characteristics of the obstacle relative to the target vehicle, providing an important basis for subsequent risk assessment and decision-making, and helping the target vehicle to better deal with obstacles in the surrounding environment and ensure driving safety.

[0073] Step S1223, calling a pre-trained lane feature encoder to perform multi-scale feature extraction on the lane curvature data sequence to generate a lane curvature matching feature, wherein the lane curvature matching feature includes a dynamic matching coefficient between the current lane curvature and the vehicle steering angle.

[0074] For example, the lane feature encoder can analyze lane curvature data at different scales, such as focusing on the local curvature of the lane at a small scale and focusing on the overall curvature change trend at a large scale. The encoder processes the lane curvature data sequence [0.008, 0.01, 0.012, 0.014, 0.016] to generate a lane curvature matching feature containing the dynamic matching coefficient of the current lane curvature and the vehicle steering angle. Assume that the generated feature value is 0.8.

[0075] Step S1224, performing sliding window statistical processing on the ambient light intensity data sequence to generate a visibility influence coefficient feature, wherein the visibility influence coefficient feature includes an associated weight parameter of the light intensity change rate and the obstacle recognition confidence.

[0076] Assume that the sliding window size is 3. For the ambient light intensity data sequence [450, 480, 510, 540, 570], the data of the first sliding window is 450, 480, 510. To calculate the rate of change of light intensity, first calculate the difference between adjacent data, 480-450=30, 510-480=30, and the average rate of change is (30+30) / 2=30. At the same time, according to the set algorithm combined with the obstacle recognition confidence, assuming that the association weight parameter calculated by the empirical formula is 0.6, the generated visibility influence coefficient feature is [30, 0.6].

[0077] Step S1225, spatially encoding and fusing the obstacle relative motion trajectory feature, lane line curvature matching feature and visibility influence coefficient feature to obtain the environment interaction feature.

[0078] For example, the obstacle relative motion trajectory feature [0.5, 0.4], the lane curvature matching feature 0.8 and the visibility influence coefficient feature [30, 0.6] are fused according to the set rules. It is assumed that the fused environmental interaction feature is [0.5, 0.4, 0.8, 30, 0.6].

[0079] Step S123, performing behavior pattern recognition processing on the driving behavior data sequence to generate behavior abnormality characteristics, wherein the behavior abnormality characteristics include steering wheel operation frequency characteristics, pedal switching interval characteristics, and line of sight deviation duration characteristics.

[0080] In a possible implementation, step S123 includes: Step S1231, collecting a steering wheel angle change data sequence, a pedal position switching data sequence, and a driver's line of sight direction data sequence.

[0081] For example, sensors installed on the steering wheel, pedals, and in front of the driver's seat are used to collect a steering wheel angle change data sequence, a pedal position switch data sequence, and a driver's line of sight direction data sequence.

[0082] Step S1232, extracting frequency domain features from the steering angle change data sequence to generate steering wheel operation frequency features, wherein the steering wheel operation frequency features include energy proportion parameters of a set frequency band.

[0083] In a possible implementation, step S1232 includes: Step S1232-1, perform denoising preprocessing on the original corner signal, use a wavelet threshold denoising algorithm to remove high-frequency noise components, and obtain a denoised corner signal.

[0084] Step S1232-2, performing short-time Fourier transform on the denoised corner signal to generate a time-frequency energy distribution matrix.

[0085] Step S1232-3, extracting energy peak features within a set frequency band of the time-frequency energy distribution matrix, and calculating a parameter of the ratio of energy within the set frequency band to total energy based on the energy peak features.

[0086] Step S1232-4, performing sliding average processing on the proportional parameters of N consecutive time windows to generate the steering wheel operation frequency characteristics.

[0087] The steering wheel angle sensor records the steering wheel angle changes in real time and generates an angle change data sequence at intervals of 0.1 seconds, such as [8, 10, 12, 14, 16] (unit: degree). The angle change data sequence is pre-processed for denoising, and the wavelet threshold denoising algorithm is used to remove high-frequency noise components. The wavelet threshold denoising algorithm first decomposes the original angle signal into wavelet coefficients of different frequencies, and then retains the wavelet coefficients above the threshold and sets the wavelet coefficients below the threshold to zero according to the set threshold. The signal is then reconstructed through the inverse wavelet transform to obtain the denoised angle signal. Assume that the denoised angle signal is [8.2, 10.1, 12.1, 14.1, 16.1].

[0088] In this embodiment, the short-time Fourier transform converts the time domain signal into a representation of the time-frequency domain, and the signal is segmented by selecting a suitable window function, and then each segment is Fourier transformed to obtain a time-frequency energy distribution matrix. For example, in the time-frequency energy distribution matrix, the frequency band is set to [15-25Hz], and the energy peak feature is extracted within the frequency band. Assume that the energy peaks found in the frequency band are [10, 12, 14, 16, 18], and then calculate the ratio parameter of the energy of the set frequency band to the total energy based on the above energy peak feature. The total energy is calculated as the sum of the energies of all frequency bands. Assuming that the total energy is 100, the first ratio parameter is 10 / 100=0.1, the second is 12 / 100=0.12, and so on. The ratio parameters of four consecutive time windows are processed by sliding average to generate the steering wheel operation frequency feature. For example, after sliding average, the steering wheel operation frequency feature is obtained as [0.11, 0.13, 0.15, 0.17].

[0089] Step S1233: performing event interval analysis on the pedal position switching data sequence to generate a pedal switching interval feature, wherein the pedal switching interval feature includes a time interval distribution feature of alternating operations of an accelerator pedal and a brake pedal.

[0090] The pedal position sensor monitors the pedal position switching, generates a pedal position switching data sequence, and records the time of each pedal switching, such as [0.4, 0.8, 1.2, 1.6, 2.0] (unit: seconds, indicating the time interval from the last switch to this switch). The pedal position switching data sequence is analyzed for event intervals to generate pedal switching interval features. For example, the time interval distribution feature of the alternating operation of the accelerator pedal and the brake pedal is obtained by analysis. Assume that the obtained feature value is [0.4, 0.4, 0.4, 0.4].

[0091] Step S1234, performing continuous deviation detection on the line of sight direction data sequence to generate a line of sight deviation duration feature, wherein the line of sight deviation duration feature includes a correlation coefficient between the duration of a single line of sight deviation event and the lateral position offset of the vehicle.

[0092] The driver's line of sight monitoring device tracks the driver's line of sight through a camera, generates a line of sight direction data sequence, and records the duration of each line of sight deviation from the front of the vehicle and the lateral position offset of the vehicle, such as [(2, 0.08), (3, 0.1), (4, 0.12), (5, 0.14), (6, 0.16)] (unit: seconds and meters). The line of sight direction data sequence is subjected to continuous deviation detection to generate a line of sight deviation duration feature, such as the correlation coefficient between the duration of a single line of sight deviation event and the lateral position offset of the vehicle. Assume that the calculated correlation coefficient is [0.4, 0.33, 0.3, 0.28, 0.25].

[0093] Step S1235, standardize and splice the steering wheel operation frequency feature, the pedal switching interval feature and the line of sight deviation duration feature to obtain the behavior abnormality feature.

[0094] In this embodiment, standardization is to map the characteristic values ​​of the steering wheel operation frequency feature, the pedal switching interval feature and the line of sight deviation duration feature to a specific range, such as the [0, 1] interval, and then splice them together in order. Assume that the standardized steering wheel operation frequency feature is [0.2, 0.3, 0.4, 0.5], the pedal switching interval feature is [0.3, 0.3, 0.3, 0.3], and the line of sight deviation duration feature is [0.2, 0.2, 0.2, 0.2], and the behavior abnormality feature obtained after splicing is [0.2, 0.3, 0.4, 0.5, 0.3, 0.3, 0.3, 0.3, 0.2, 0.2, 0.2].

[0095] Step S124, performing spatiotemporal alignment and fusion processing on the vehicle movement trend feature, the environment interaction feature and the behavior abnormality feature to generate the dynamic spatiotemporal coding feature.

[0096] In this embodiment, spatiotemporal alignment is to match and calibrate different features in time and space dimensions to ensure that they can be fused in the same dimension. For example, the vehicle motion trend feature is [0.1, 0.1, 0.1, 0.0067, 0.008, 0.009, 0.01, 2, 2, 2, 2], the environment interaction feature is [0.5, 0.4, 0.8, 30, 0.6], and the behavior abnormality feature is [0.2, 0.3, 0.4, 0.5, 0.3, 0.3, 0.3, 0.3, 0.2, 0.2, 0.2, 0.2].

[0097] When performing fusion, the above feature vectors are first dimensionally unified. Since the dimensions of different feature vectors may be different, they need to be made consistent by padding or dimensionality reduction. Here, it is assumed that the environment interaction feature vector is padded with zeros to the same length as the other two feature vectors. For example, the environment interaction feature after padding is [0.5, 0.4, 0.8, 30, 0.6, 0, 0, 0, 0, 0, 0, 0].

[0098] Then, the three feature vectors are fused according to the set rules. For example, a simple splicing method can be used to splice the vehicle movement trend feature, the filled environment interaction feature and the behavior abnormality feature together in sequence to obtain the dynamic spatiotemporal coding feature of [0.1, 0.1, 0.1, 0.1, 0.0067, 0.008, 0.009, 0.01, 2, 2, 2, 2, 0.5, 0.4, 0.8, 30, 0.6, 0, 0, 0, 0, 0, 0, 0.2, 0.3, 0.4, 0.5, 0.3, 0.3, 0.3, 0.2, 0.2, 0.2, 0.2].

[0099] This dynamic spatiotemporal coding feature comprehensively integrates information from various aspects such as vehicle operation status, environmental perception, and driving behavior, and integrates data features from different sources into one vector, providing rich and valuable input for the subsequent invocation of the pre-trained collision risk prediction model for multi-dimensional risk association analysis and processing, so that the collision risk prediction model can more accurately analyze and predict the various risk conditions currently faced by the vehicle based on the above comprehensive features. In this way, the operation status of the vehicle in a complex environment can be better understood, thereby providing a solid data foundation for the formulation of real-time warning strategies and risk intervention operations, and ultimately ensuring the safe driving of the vehicle.

[0100] In a possible implementation, step S130 includes: Step S131, inputting the dynamic spatiotemporal coding features into the spatiotemporal attention module of the collision risk prediction model to generate feature weight distributions of different time windows.

[0101] For example, in a possible implementation, step S131 includes: Step S1311, input the dynamic spatiotemporal coding features of the current time window and the dynamic spatiotemporal coding features of the historical time window into the linear mapping layer of the spatiotemporal attention module, and generate a current query feature vector, a historical key feature vector and a historical value feature vector respectively, wherein the historical key feature vector and the historical value feature vector are arranged in the order of the time window.

[0102] For example, in this process, the dynamic spatiotemporal coding features of the current time window and the dynamic spatiotemporal coding features of the historical time window are input into the linear mapping layer of the spatiotemporal attention module. Assume that the dynamic spatiotemporal coding features of the current time window are a vector of length 50, and each element in the vector represents a feature value of different aspects. For example, the first 10 elements represent the acceleration change rate related feature values ​​in the vehicle motion trend feature, the middle 20 elements represent the obstacle relative motion trajectory and other related feature values ​​in the environmental interaction feature, and the last 20 elements represent the steering wheel operation frequency and other related feature values ​​in the behavior abnormality feature. The historical time window is set to contain the dynamic spatiotemporal coding features of the first three time windows, and each feature vector is also 50 in length.

[0103] After being processed by the linear mapping layer, the current query feature vector, historical key feature vector and historical value feature vector are generated respectively. The linear mapping layer realizes mapping by performing matrix multiplication operation with the input feature vector through a specific weight matrix. For example, for the dynamic spatiotemporal coding feature vector of the current time window, the weight matrix in the linear mapping layer is a 50×30 matrix. Through matrix multiplication operation, the 50-dimensional feature vector is mapped to the 30-dimensional current query feature vector. For the dynamic spatiotemporal coding feature vector of the historical time window, similar linear mapping operations are also performed to generate the historical key feature vector and the historical value feature vector arranged in the order of the time window. Here, the historical key feature vector and the historical value feature vector are also 30-dimensional vectors.

[0104] Step S1312: Generate an initial time window relevance sequence by performing a dot product operation on the current query feature vector and the key feature vector of each historical time window.

[0105] In this embodiment, the dot product operation is to multiply the elements of the corresponding positions of the two vectors and then sum them. Assuming that the current query feature vector is [q1, q2, ..., q30], and the key feature vector of the first historical time window is [k11, k12, ..., k130], then their dot product operation is q1×k11+q2×k12+...+q30×k130, and a scalar value is obtained, which is the initial correlation between the current time window and the first historical time window. In the same way, the dot product of the current query feature vector and the key feature vectors of the second and third historical time windows is calculated in turn to generate an initial time window correlation sequence. Assume that the obtained sequence is [0.2, 0.3, 0.4].

[0106] Step S1313, performing an exponential scaling operation of a learnable scaling coefficient on the initial time window relevance sequence to generate an attention energy distribution before normalization.

[0107] Here, the learnable scaling factor is a parameter learned by the model during training. Assume that the current learnable scaling factor is 1.5. For each value in the initial time window association sequence, an exponential scaling operation is performed, that is, the first value 0.2 is calculated as e to the power of (0.2×1.5), where e is a natural constant of approximately 2.718. Following the same calculation method, 0.3 and 0.4 in the sequence are calculated to obtain the attention energy distribution before normalization, which is assumed to be [1.34, 2.01, 2.71].

[0108] Step S1314, performing Softmax normalization calculation on the attention energy distribution along the time window dimension to generate a weight distribution coefficient for each historical time window.

[0109] In this embodiment, the calculation process of Softmax normalization is to calculate the result of dividing each value in the attention energy distribution by the sum of all values. For example, for the above attention energy distribution [1.34, 2.01, 2.71], the sum is 1.34+2.01+2.71=6.06. Then the weight distribution coefficient of the first historical time window is 1.34÷6.06≈0.22, the weight distribution coefficient of the second historical time window is 2.01÷6.06≈0.33, and the weight distribution coefficient of the third historical time window is 2.71÷6.06≈0.45. In this way, a sequence of weight distribution coefficients [0.22, 0.33, 0.45] for each historical time window is generated.

[0110] Step S1315, performing a window-by-window weighted superposition operation on the weight allocation coefficient and the historical value feature vector in the order of the time windows to generate feature weight distributions for different time windows, wherein the dimension of the feature weight distribution is consistent with the number of historical time windows and each element corresponds to the weight value of a single window.

[0111] Taking the first historical time window as an example, the weight distribution coefficient is 0.22, the historical value feature vector is [v11, v12, …, v130], and the weighted vector is [0.22×v11, 0.22×v12, …, 0.22×v130]. ​​In the same way, the historical value feature vectors of the second and third historical time windows are weighted, and then the weighted vectors are superimposed in the order of the time windows to obtain the feature weight distribution of different time windows. Here, the dimension of the feature weight distribution is consistent with the number of historical time windows, which is 3, and each element corresponds to the weight value of a single window. For example, the feature weight distribution obtained is [[0.22×v11, 0.22×v12, …, 0.22×v130], [0.33×v21, 0.33×v22, …, 0.33×v230], [0.45×v31, 0.45×v32, …, 0.45×v330]].

[0112] Step S132, dynamically weighted fusion of the features of the historical time window is performed based on the feature weight distribution to generate an enhanced spatiotemporal feature representation.

[0113] In this embodiment, dynamic weighted fusion is to perform weighted summation of the feature weight distribution of different time windows obtained above and the original features of the corresponding historical time windows. Taking the first element as an example, assuming that the element of the original feature vector of the first historical time window at a certain position is f1, the element of the original feature vector of the second historical time window at the same position is f2, and the element of the original feature vector of the third historical time window at the same position is f3, and the corresponding weight distribution coefficients are 0.22, 0.33, and 0.45, then the enhanced feature value at this position after fusion is 0.22×f1+0.33×f2+0.45×f3. According to the same method, the elements at all positions are calculated to obtain a new vector, which is the enhanced spatiotemporal feature representation, which integrates the feature information of different time windows, and reasonably weights the features of different time windows according to the feature weight distribution, highlighting the feature information of historical time windows with a higher correlation with the current time window.

[0114] Step S133, inputting the enhanced spatiotemporal feature representation into the risk classifier of the collision risk prediction model, wherein the risk classifier comprises a longitudinal risk branch, a lateral risk branch and an emergency braking branch arranged in parallel. The longitudinal risk branch outputs a longitudinal collision risk level, the lateral risk branch outputs a lateral deviation risk level, and the emergency braking branch outputs an emergency braking trigger probability.

[0115] For example, there are multiple neural network layers inside the longitudinal risk branch, which deeply analyze the features related to longitudinal motion in the enhanced spatiotemporal feature representation. The above features may include the longitudinal acceleration change rate of the vehicle, the longitudinal distance change trend to the obstacle in front, etc. Through the calculation of the neural network layer, the above features are first weighted and combined, then transformed nonlinearly, and finally the longitudinal collision risk level is judged according to the preset threshold. Assume that after calculation, when the result after weighted combination and nonlinear transformation is greater than a certain high risk threshold, the output longitudinal collision risk level is high; when the result is within the medium risk threshold range, the output longitudinal collision risk level is medium; when the result is less than the low risk threshold, the output longitudinal collision risk level is low.

[0116] The lateral risk branch focuses on analyzing features related to lateral departure risk, such as changes in the vehicle's lateral angular velocity, lateral distance from the lane line, and other features. The lateral departure risk level is also generated through calculations and judgments made through the internal neural network layer. For example, the neural network layer can extract and analyze the above features, calculate the relationship and change trend between the features, and judge the lateral departure risk level according to the set rules. If the vehicle's lateral angular velocity changes greatly and the lateral distance from the lane line is close to the dangerous range, the neural network layer may output a high lateral departure risk level after calculation; if the above features are within a certain safety range, the corresponding medium or low risk level is output.

[0117] The emergency braking branch mainly outputs the emergency braking trigger probability based on some key features in the enhanced spatiotemporal feature representation, such as the vehicle's brake pressure changes, the approaching speed of the obstacle in front, etc. The emergency braking branch processes the above features through a set algorithm, such as performing a trend analysis on the brake pressure changes, and calculating the emergency braking trigger probability through a probability model in combination with the approaching speed of the obstacle in front. Assume that after a series of calculations, when the brake pressure rises rapidly and the approaching speed of the obstacle in front exceeds a certain critical value, the probability of emergency braking triggering calculated by the probability model is 0.6; if the above conditions are relatively less urgent, the calculated probability may be reduced accordingly.

[0118] Step S134, performing decision-level fusion on the output results of the longitudinal risk branch, the lateral risk branch and the emergency braking branch arranged in parallel to generate the risk prediction result.

[0119] In this embodiment, decision-level fusion can adopt a variety of methods, such as a simple voting method or a weighted average method. Assuming the weighted average method is adopted, the longitudinal collision risk level output by the longitudinal risk branch is expressed by a numerical value, with a high risk of 3, a medium risk of 2, and a low risk of 1; the lateral deviation risk level output by the lateral risk branch is also expressed by a numerical value, with a high risk of 3, a medium risk of 2, and a low risk of 1; the emergency braking trigger probability output by the emergency braking branch remains unchanged. Different weights are set for these three branches, assuming that the weight of the longitudinal risk branch is 0.4, the weight of the lateral risk branch is 0.3, and the weight of the emergency braking branch is 0.3.

[0120] For the longitudinal collision risk level, it is assumed that the output is medium risk, that is, the value 2; the lateral risk branch outputs a low risk, that is, the value 1; and the emergency braking branch outputs an emergency braking trigger probability of 0.6. First, the weighted value of the longitudinal risk branch is calculated as 2×0.4=0.8, the weighted value of the lateral risk branch is 1×0.3=0.3, and the weighted value of the emergency braking branch is 0.6×0.3=0.18. Then these three weighted values ​​are added together to get a total of 0.8+0.3+0.18=1.28. According to the pre-set rules, the sum is mapped to the corresponding risk level and probability range, for example, when the sum is within a certain range, it corresponds to a certain comprehensive risk condition. Assuming that the range of 1-1.5 corresponds to the risk prediction result of "medium and low risk, low possibility of emergency braking", the final risk prediction result clearly informs the comprehensive situation of the longitudinal collision risk, lateral deviation risk, and emergency braking trigger probability currently faced by the vehicle, providing an accurate basis for the subsequent construction of a real-time warning strategy set based on the result, so that the vehicle can take corresponding measures in time to ensure driving safety.

[0121] Figure 2 A schematic diagram of exemplary hardware and software components of an artificial intelligence-based vehicle collision warning system 100 that can implement the concept of the present invention is shown in some embodiments of the present invention. For example, the processor 120 can be used in the artificial intelligence-based vehicle collision warning system 100 and is used to perform the functions of the present invention.

[0122] The artificial intelligence-based vehicle collision warning system 100 can be a general-purpose server or a special-purpose server, both of which can be used to implement the artificial intelligence-based vehicle collision warning method of the present invention. Although the present invention only shows one server, for convenience, the functions described in the present invention can be implemented in a distributed manner on multiple similar platforms to balance the processing load.

[0123] For example, the artificial intelligence-based vehicle collision warning system 100 may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and storage media 140 in different forms, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the artificial intelligence-based vehicle collision warning system 100 may also include program instructions stored in ROM, RAM, or other types of non-temporary storage media, or any combination thereof. The method of the present invention can be implemented according to these program instructions. The artificial intelligence-based vehicle collision warning system 100 also includes an input / output (I / O) interface 150 between the computer and other input / output devices.

[0124] For ease of explanation, only one processor is described in the artificial intelligence-based vehicle collision warning system 100. However, it should be noted that the artificial intelligence-based vehicle collision warning system 100 in the present invention may also include multiple processors, so the steps performed by one processor described in the present invention may also be performed jointly or individually by multiple processors. For example, if the processor of the artificial intelligence-based vehicle collision warning system 100 performs step A and step B, it should be understood that step A and step B may also be performed jointly by two different processors or individually in one processor. For example, the first processor performs step A, the second processor performs step B, or the first processor and the second processor perform steps A and B together.

[0125] In addition, an embodiment of the present invention further provides a readable storage medium, in which computer executable instructions are preset. When a processor executes the computer executable instructions, the above-mentioned vehicle collision warning method based on artificial intelligence is implemented.

[0126] It should be noted that in order to simplify the description of the present invention and thus help understand one or more embodiments of the invention, in the foregoing description of the embodiments of the present invention, various features are sometimes combined into one embodiment, figure or description thereof.

Claims

1. A vehicle collision warning method based on artificial intelligence, characterized in that: The method comprises: Acquire a multi-source sensor data set of a target vehicle, wherein the multi-source sensor data set includes a vehicle operation status data sequence, an environment perception data sequence, and a driving behavior data sequence, each data sequence consisting of standardized monitoring data of multiple continuous time windows; Performing a dynamic spatiotemporal feature extraction operation on the multi-source sensor data set to generate dynamic spatiotemporal coding features for each time window, wherein the dynamic spatiotemporal coding features include vehicle motion trend features, environmental interaction features, and behavior abnormality features; Calling a pre-trained collision risk prediction model to perform multi-dimensional risk association analysis on the dynamic spatiotemporal coding features to generate a risk prediction result for the current time window, wherein the risk prediction result includes a longitudinal collision risk level, a lateral deviation risk level, and an emergency braking trigger probability; Building a real-time warning strategy set based on the risk prediction result, the real-time warning strategy set includes warning signal generation rules and vehicle control instruction sequences corresponding to different risk levels; The real-time warning strategy set is deployed to the vehicle-mounted decision-making system, and a control instruction mapping relationship with the vehicle actuator is established to trigger real-time risk intervention operations.

2. The vehicle collision warning method based on artificial intelligence according to claim 1 is characterized in that: The performing of a dynamic spatiotemporal feature extraction operation on the multi-source sensor data set to generate a dynamic spatiotemporal coding feature for each time window includes: Performing motion feature extraction processing on the vehicle running state data sequence to obtain vehicle motion trend features, wherein the vehicle motion trend features include acceleration change rate features, steering angular velocity features, and brake pressure gradient features; Performing spatial correlation analysis on the environment perception data sequence to extract environment interaction features, wherein the environment interaction features include obstacle relative motion trajectory features, lane line curvature matching degree features, and visibility influence coefficient features; Performing behavior pattern recognition processing on the driving behavior data sequence to generate behavior abnormality characteristics, wherein the behavior abnormality characteristics include steering wheel operation frequency characteristics, pedal switching interval characteristics, and line of sight deviation duration characteristics; The vehicle movement trend feature, the environment interaction feature and the behavior abnormality feature are subjected to spatiotemporal alignment and fusion processing to generate the dynamic spatiotemporal coding feature.

3. The vehicle collision warning method based on artificial intelligence according to claim 2 is characterized in that: The step of performing motion feature extraction processing on the vehicle running status data sequence to obtain vehicle motion trend features includes: Collecting a longitudinal acceleration data sequence, a lateral angular velocity data sequence and a brake pressure data sequence of the target vehicle; Constructing multiple parallel time convolution networks to process the longitudinal acceleration data sequence, the lateral angular velocity data sequence and the brake pressure data sequence respectively, and generating acceleration timing features, steering timing features and braking timing features; Performing differential calculation on the acceleration time series characteristics to obtain acceleration change rate characteristics, performing sliding window variance calculation on the steering time series characteristics to obtain steering angular velocity fluctuation characteristics, and performing gradient analysis on the braking time series characteristics to obtain braking pressure gradient characteristics; The acceleration change rate feature, the steering angular velocity fluctuation feature and the brake pressure gradient feature are feature spliced ​​to obtain the vehicle motion trend feature.

4. The vehicle collision warning method based on artificial intelligence according to claim 3 is characterized in that: The method of constructing a plurality of parallel time convolution networks to process the longitudinal acceleration data sequence, the lateral angular velocity data sequence and the brake pressure data sequence respectively to generate acceleration timing features, steering timing features and brake timing features includes: Constructing a first time convolutional network for the longitudinal acceleration data sequence, wherein the first time convolutional network comprises alternating dilated convolutional layers and normalization layers for capturing multi-scale acceleration pattern features; Constructing a second time convolution network for the lateral angular velocity data sequence, wherein the second time convolution network comprises a bidirectional convolution layer and a gated activation unit, and is used to extract the timing-dependent features of the steering operation; Constructing a third temporal convolutional network for the brake pressure data sequence, wherein the third temporal convolutional network comprises a residual convolution module and an attention pooling layer, and is used to identify the sudden characteristics of the braking behavior; The outputs of the first time convolution network, the second time convolution network and the third time convolution network are respectively subjected to feature dimensionality reduction processing to obtain the acceleration timing features, the steering timing features and the braking timing features.

5. The vehicle collision warning method based on artificial intelligence according to claim 2 is characterized in that: The performing of spatial correlation analysis on the environment perception data sequence to extract environment interaction features, wherein the environment interaction features include obstacle relative motion trajectory features, lane line curvature matching degree features, and visibility influence coefficient features, including: Obtaining a relative position data sequence of obstacles around the target vehicle, a lane line curvature data sequence, and an ambient light intensity data sequence; Constructing an obstacle motion trajectory prediction model, estimating the motion state of the relative position data sequence based on a Kalman filter algorithm, and generating obstacle relative motion trajectory features, wherein the obstacle relative motion trajectory features include lateral displacement prediction error features and longitudinal velocity covariance features; Calling a pre-trained lane feature encoder to perform multi-scale feature extraction on the lane curvature data sequence to generate a lane curvature matching feature, wherein the lane curvature matching feature includes a dynamic matching coefficient between the current lane curvature and the vehicle steering angle; Performing sliding window statistical processing on the ambient light intensity data sequence to generate a visibility influence coefficient feature, wherein the visibility influence coefficient feature includes an associated weight parameter of the light intensity change rate and the obstacle recognition confidence; The obstacle relative motion trajectory feature, lane line curvature matching feature and visibility influence coefficient feature are spatially encoded and fused to obtain the environmental interaction feature.

6. The vehicle collision warning method based on artificial intelligence according to claim 5 is characterized in that: The step of constructing an obstacle motion trajectory prediction model and estimating the motion state of the relative position data sequence based on a Kalman filter algorithm to generate obstacle relative motion trajectory features includes: Initializing a motion state vector of the obstacle, wherein the motion state vector includes a lateral position observation value, a longitudinal velocity observation value, and an acceleration estimation value; Generate a state transfer matrix and an observation noise covariance matrix based on the vehicle coordinate system, perform time step iterative prediction on the motion state vector through the state transfer matrix, and output the predicted state vector and the corresponding predicted covariance matrix; Perform residual calculation on the predicted state vector and the actual observation value of the current time window, and calculate the Mahalanobis distance of the residual vector in combination with the predicted covariance matrix; When the Mahalanobis distance exceeds a dynamic adjustment threshold, the noise parameters of the observation noise covariance matrix are corrected by a scaling factor to generate an updated observation noise covariance matrix; The updated observation noise covariance matrix is ​​used to re-execute the measurement update step of the Kalman filter, and the corrected prediction state vector and lateral displacement prediction error characteristics are output; Inputting the corrected predicted state vector into a trajectory pattern matching network, performing convolution similarity comparison with typical motion patterns in a historical trajectory database, and generating longitudinal velocity covariance features; The lateral displacement prediction error feature and the longitudinal velocity covariance feature are integrated to generate the obstacle relative motion trajectory feature.

7. The vehicle collision warning method based on artificial intelligence according to claim 2 is characterized in that: The performing behavior pattern recognition processing on the driving behavior data sequence to generate behavior abnormality characteristics includes: Collecting steering wheel angle change data sequence, pedal position switching data sequence and driver's sight direction data sequence; Extracting frequency domain features of the steering angle change data sequence to generate steering wheel operation frequency features, wherein the steering wheel operation frequency features include energy proportion parameters of a set frequency band; Performing event interval analysis on the pedal position switching data sequence to generate a pedal switching interval feature, wherein the pedal switching interval feature includes a time interval distribution feature of alternating operations of an accelerator pedal and a brake pedal; Performing continuous deviation detection on the sight direction data sequence to generate a sight deviation duration feature, wherein the sight deviation duration feature includes a correlation coefficient between the duration of a single sight deviation event and a lateral position offset of the vehicle; The steering wheel operation frequency feature, the pedal switching interval feature and the line of sight deviation duration feature are standardized and spliced ​​to obtain the behavior abnormality feature.

8. The vehicle collision warning method based on artificial intelligence according to claim 7 is characterized in that: The extracting frequency domain features of the steering angle change data sequence to generate steering wheel operation frequency features includes: The original corner signal is pre-processed for denoising, and the high-frequency noise component is removed by using the wavelet threshold denoising algorithm to obtain the denoised corner signal; Performing short-time Fourier transform on the denoised corner signal to generate a time-frequency energy distribution matrix; Extracting energy peak features within a set frequency range of the time-frequency energy distribution matrix, and calculating a parameter of the ratio of energy within the set frequency range to total energy based on the energy peak features; The proportional parameters of N consecutive time windows are processed by sliding average to generate the steering wheel operation frequency characteristics.

9. The vehicle collision warning method based on artificial intelligence according to claim 1, characterized in that: The calling of the pre-trained collision risk prediction model performs multi-dimensional risk association analysis on the dynamic spatiotemporal coding features to generate a risk prediction result for the current time window, including: Inputting the dynamic spatiotemporal coding features into the spatiotemporal attention module of the collision risk prediction model to generate feature weight distributions of different time windows; Based on the feature weight distribution, the features of the historical time window are dynamically weighted and fused to generate an enhanced spatiotemporal feature representation; The enhanced spatiotemporal feature representation is input into a risk classifier of the collision risk prediction model, wherein the risk classifier comprises a longitudinal risk branch, a lateral risk branch and an emergency braking branch arranged in parallel; wherein the longitudinal risk branch outputs a longitudinal collision risk level, the lateral risk branch outputs a lateral deviation risk level, and the emergency braking branch outputs an emergency braking trigger probability; The output results of the longitudinal risk branch, the lateral risk branch and the emergency braking branch arranged in parallel are fused at the decision level to generate the risk prediction result.

10. A vehicle collision warning system based on artificial intelligence, characterized in that: The artificial intelligence-based vehicle collision warning system includes a processor and a memory, the memory is connected to the processor, the memory is used to store programs, instructions or codes, and the processor is used to execute the programs, instructions or codes in the memory to implement the artificial intelligence-based vehicle collision warning method described in any one of claims 1 to 9 above.

Images