End-to-end driving style identification method and system based on multi-source sensing data

Abstract

The invention provides an end-to-end driving style identification method and system based on multi-source sensing data, and relates to the technical field of driving style identification, and the method comprises the steps: obtaining a historical driving record and a driving style corresponding to the historical driving record; performing preprocessing including data synchronization, feature selection, data standardization and data segmentation on the data set to obtain a training set, a verification set and a test set in a preset proportion; constructing an MAK model; inputting the training set into the MAK model to train the MAK model; acquiring a real-time driving record; and inputting a real-time driving record into the trained MAK model, and outputting a predicted driving style. According to the invention, the complex spatial-temporal characteristics of the driving behavior can be accurately captured, and efficient and real-time driving style recognition is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of driving style recognition, and in particular to an end-to-end driving style recognition method and system based on multi-source sensor data. Background Art

[0002] The end-to-end driving style recognition method based on multi-source sensor data is a deep learning technology that combines multiple sensor data (such as acceleration, gyroscope, GPS, etc.) to automatically identify the driver's driving style. This method learns directly from the sensor data and generates driving behavior classification results without the need for complex manual feature extraction. This type of method can improve driving safety, improve user experience, and support the application of autonomous driving and driving behavior analysis.

[0003] With the development of artificial intelligence technology, sensor technology, Internet technology, big data and 5G network, smart cars based on electrification, intelligence and networking have become the development trend of the automotive industry. A potential but clear direction for smart vehicles is to shift from the initial faster and more fuel-efficient direction to a more humane and personalized direction. Driving style is a summary of the driver's driving behavior and habits, reflecting the driver's driving characteristics. It has positive significance in the fields of driving safety, automobile intelligence, automobile energy efficiency and automobile insurance industry. Accurately identifying the driver's driving style is a very valuable research work.

[0004] However, when processing multivariate long-term series data, traditional driving style recognition methods have problems such as insufficient global feature extraction capabilities and poor performance with small samples. They are unable to capture complex driving behaviors, are inefficient and difficult to adapt to changing driving scenarios. They use short time windows to identify driving styles, which are easily affected by instantaneous behaviors and environmental changes, resulting in unstable recognition results and difficulty in truly reflecting the driver's long-term driving habits and personality. Summary of the invention

[0005] In order to solve the technical problems in the prior art that traditional driving style recognition methods usually rely on manual feature extraction, which is time-consuming and cumbersome, often misses hidden and useful features, cannot capture complex driving behaviors, is inefficient, and is difficult to adapt to diverse driving scenarios, uses a short time window to identify driving style, is easily affected by instantaneous behavior and environmental changes, resulting in unstable recognition results, and is difficult to truly reflect the driver's long-term driving habits and personality, the present invention provides an end-to-end driving style recognition method and system based on multi-source sensor data.

[0006] The technical solution provided by the embodiment of the present invention is as follows:

[0007] First aspect

[0008] An embodiment of the present invention provides an end-to-end driving style recognition method based on multi-source sensor data, comprising:

[0009] S1: Acquire historical driving records and driving styles corresponding to the historical driving records, wherein the historical driving records and the driving styles constitute a data set, wherein the historical driving records include GPS signal data, inertial measurement signal data, and vehicle detection data;

[0010] S2: Preprocess the data set including data synchronization, feature selection, data standardization and data segmentation to obtain training sets, validation sets and test sets with preset proportions;

[0011] S3: Construct a MAK model, wherein the MAK model includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module and an output module connected in sequence, the Mamba module includes a first channel and a second channel, wherein the first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit and a second projection unit connected in sequence, the second channel includes a third projection unit and a second activation function unit, wherein the second channel is connected to the channel fusion unit, and the first channel and the second channel have the same input data;

[0012] S4: input the training set into the MAK model to train the MAK model;

[0013] S5: Obtain real-time driving records;

[0014] S6: Input the real-time driving record into the trained MAK model and output the predicted driving style.

[0015] Second aspect

[0016] An embodiment of the present invention provides an end-to-end driving style recognition system based on multi-source sensor data, comprising:

[0017] processor;

[0018] A memory having computer-readable instructions stored therein, wherein when the computer-readable instructions are executed by the processor, the end-to-end driving style recognition method based on multi-source sensor data as in the first aspect is implemented.

[0019] The third aspect

[0020] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the end-to-end driving style recognition method based on multi-source sensor data as described in the first aspect is implemented.

[0021] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0022] In the present invention, features are automatically extracted from multi-source sensor data through the MAK model without relying on manual feature extraction, thereby improving the efficiency of feature extraction, and being able to discover potential and complex features, thereby improving the accuracy of recognition. The KAN network module and the global average pooling module can extract stable and global features in long time series, which can better reflect the driver's personality and driving habits, and reduce the impact of short-term behavior fluctuations on recognition results. Through the efficient calculation of the convolutional network unit, the KAN model can run quickly in a resource-constrained environment, ensuring the real-time performance of the model. Through the dual-channel structure of the Mamba module, the model can extract features from multiple angles and integrate various types of information through channel fusion, thereby improving the stability of the model, reducing the dependence on specific data sets, and being able to accurately identify a variety of complex driving scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic flow chart of an end-to-end driving style recognition method based on multi-source sensor data provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of the structure of a MAK model provided in an embodiment of the present invention;

[0026] Figure 3 The structural intention of the Mamba module provided by the embodiment of the present invention;

[0027] Figure 4 A schematic diagram of the structure of an end-to-end driving style recognition system based on multi-source sensor data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0029] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0030] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0031] Reference Manual Attached Figure 1 , showing a flow chart of an end-to-end driving style recognition method based on multi-source sensor data provided by an embodiment of the present invention.

[0032] An embodiment of the present invention provides an end-to-end driving style recognition method based on multi-source sensor data, the method comprising:

[0033] S1: Acquire historical driving records and driving styles corresponding to the historical driving records, wherein the historical driving records and the driving styles constitute a data set, wherein the historical driving records include GPS signal data, inertial measurement signal data, and vehicle detection data.

[0034] Among them, historical driving records refer to data generated during the driving process that are collected over a long period of time through vehicle sensors, GPS, inertial measurement devices, etc., and are driving records of multiple drivers performing various driving styles on highways and secondary roads. GPS signal data is vehicle location information collected by the global positioning system (GPS), including speed, direction and geographic location. Inertial measurement signal data is data collected through inertial measurement devices such as accelerometers and gyroscopes, reflecting the vehicle's acceleration, angular velocity and other motion information. Vehicle detection data is the vehicle's sensor output information, such as vehicle speed, throttle, brakes, steering angle, distance to the vehicle in front, etc.

[0035] In a possible implementation manner, the driving styles include normal driving, slow driving and aggressive driving, and S1 is specifically:

[0036] Use the built-in sensors of the mobile phone to obtain the data set.

[0037] Among them, the built-in sensors in the mobile phone refer to the common sensor components in modern smartphones, which can capture the vehicle's motion status and environmental information. Common built-in sensors in mobile phones include accelerometers for measuring the linear acceleration of the mobile phone along different axes and reflecting the acceleration and deceleration of the vehicle, gyroscopes for measuring the angular velocity of the mobile phone and reflecting the turning and rotation behavior of the vehicle, GPS modules for collecting location information and magnetometers for measuring the earth's magnetic field.

[0038] It should be noted that by obtaining rich historical driving records and multi-source sensor data (GPS, inertial measurement, vehicle sensors, etc.), the driver's driving behavior can be fully reflected, and a high-quality data set with real driving style labels can be provided for the driving style recognition model.

[0039] S2: The data set is preprocessed including data synchronization, feature selection, data standardization and data segmentation to obtain training sets, validation sets and test sets with preset proportions.

[0040] Among them, data synchronization is to align the data from different sensors according to a unified time base. Because the sampling frequencies of different sensors may be different, data synchronization ensures that the data is consistent at the same time point. Feature selection is to extract key features useful for driving style recognition from the data set. Data standardization is to scale data with different features to the same scale. Data segmentation is to divide the preprocessed data set into training set, validation set and test set. The training set is a part of the data set specifically used to train the machine learning model. The model learns the characteristic patterns of driving style on this data. The validation set is used to monitor the model performance during the model training process and help adjust the hyperparameters to prevent the model from overfitting or underfitting. The test set is used for the final evaluation after the model training is completed to check the performance of the model on unseen data.

[0041] In a possible implementation, data synchronization specifically includes:

[0042] The frequency of collecting GPS signal data and vehicle measurement data is used as the sampling frequency.

[0043] The sampling frequency is the rate at which data is collected. Usually, different sensors have different sampling frequencies.

[0044] It should be noted that by clearly collecting the frequency of GPS signals and vehicle measurement data, the time accuracy of the data collection process can be ensured and sufficient details can be captured, especially in rapidly changing driving scenarios.

[0045] The sampling frequency with the highest frequency among the sampling frequencies is selected as the target frequency.

[0046] The sampling timestamps in the GPS signal data and vehicle detection data are extracted respectively.

[0047] The sampling timestamp is the time stamp when each data point is collected, which records the collection time of the data at a specific time point.

[0048] It should be noted that extracting the sampling timestamps from the GPS signal data and the vehicle detection data separately can ensure the temporal consistency of the data, avoid analysis errors caused by time differences, and ensure the accuracy of identifying driving behavior.

[0049] According to the target frequency and sampling timestamp, the GPS signal data is interpolated using the linear interpolation function, and the vehicle detection data is interpolated using the nearest neighbor interpolation function, so that the GPS signal data and the vehicle detection data are synchronized to the same time series:

[0050]

[0051] y(t)=y i

[0052] Among them, x(t) represents the interpolation result at time t, x i represents the i-th known data point collected, x i+1 represents the i+1th known data point collected, t i Represents a known data point x i The corresponding time, t i+1 Represents a known data point x i+1 The corresponding time, t i <t<t i+1 , y(t) represents the nearest neighbor interpolation at time t, y i Represents the discrete data value at time t calculated by the nearest neighbor interpolation function.

[0053] In a possible implementation, feature selection is specifically as follows:

[0054] Features in GPS signal data, inertial measurement signal data and vehicle detection data are selected as feature sets, wherein the features in the GPS signal data include vehicle speed and vehicle heading change, and the features in the inertial measurement signal data include acceleration of each axis of the sensor, jerk of each axis of the sensor, sensor flip angle, sensor pitch angle, sensor yaw angle and acceleration of each axis of the sensor after Kalman filtering.

[0055] Among them, the vehicle speed indicates the driving speed of the vehicle, which is used to reflect the acceleration and deceleration behavior in the driving style. The vehicle heading change is the change in the steering direction of the vehicle during driving, usually expressed in angles, which can reflect the steering habits of the driver. The inertial measurement signal data is the vehicle motion state data collected by inertial measurement units (IMUs) such as accelerometers and gyroscopes, reflecting the acceleration and rotational motion of the vehicle. The acceleration of each axis of the sensor is the linear acceleration along the different axes (X, Y, Z) of the vehicle, which is used to measure the acceleration, braking, turning and other behaviors of the vehicle. The jerk of each axis of the sensor is the rate of change of acceleration, indicating the sudden change of vehicle acceleration or deceleration, reflecting the abruptness of driving behavior. The sensor flip angle is the angle at which the vehicle flips along its longitudinal axis, which is used to measure the roll of the vehicle when turning. The sensor pitch angle is the angle at which the vehicle tilts up and down along the lateral axis, which is usually used to detect the uphill and downhill conditions of the vehicle. The sensor yaw angle is the angle at which the vehicle rotates around the vertical axis, reflecting the turning and heading changes of the vehicle. Kalman filtering is a signal processing technology used to remove noise from sensor data to generate smoother and more accurate acceleration data.

[0056] It should be noted that by selecting key features from different sources, the driver's driving behavior pattern can be fully and accurately reflected, further improving the accuracy and robustness of the data, reducing noise interference, providing high-quality input for the driving style recognition model, and improving recognition accuracy.

[0057] The jerk of each axis of the sensor is calculated based on the acceleration of each axis in the inertial measurement signal data:

[0058]

[0059] Among them, j t represents the jerk at time t, a t represents the acceleration at time t, a t-1 represents the acceleration at time t-1, and dt represents the time difference between time t and time t-1.

[0060] Features in the vehicle detection data include the distance to the preceding vehicle in the current lane, the time to collision with the preceding vehicle, the number of vehicles detected in the vehicle detection data, and the road type.

[0061] In a possible implementation, data standardization is specifically as follows:

[0062] Standardize the feature set to a value with a mean of 0 and a standard deviation of 1:

[0063]

[0064] Among them, z t represents the standardized value at time t, xt represents the feature data value at time t, μ represents the mean value of the feature vector, and σ represents the standard deviation of the feature vector.

[0065] Among them, the mean refers to the central value of the data set, which indicates the central tendency of the data points. In the process of standardization, the mean of the data points is set to 0 to eliminate the offset. The standard deviation is the degree of dispersion of the data set, which indicates the degree of deviation of the data points from the mean. In the process of standardization, the standard deviation is set to 1 to eliminate the dimensional differences between the features and make their distribution consistent.

[0066] It should be noted that by standardizing the feature set, the scale differences between different features are eliminated, making the impact of the features on the model more balanced, which helps to improve the convergence speed of the model, avoid some features with larger dimensions from having too much impact on the model, and improve the prediction accuracy and stability of the model.

[0067] In a possible implementation, data segmentation specifically includes:

[0068] According to the time sequence, the standardized feature set is segmented using a sliding time window to obtain training sets, validation sets, and test sets with preset proportions.

[0069] It should be noted that data quality is ensured and model performance is optimized through data synchronization, feature selection, data standardization and data segmentation. Data synchronization eliminates the difference in sampling time of different sensors, feature selection improves recognition efficiency, standardization reduces dimensionality effects, and data segmentation ensures the generalization ability of the model, making it stable on new data.

[0070] Reference Manual Attached Figure 2 , which shows a schematic diagram of the structure of the MAK model provided in an embodiment of the present invention.

[0071] like Figure 2 The MAK model includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module, and an output module, which are connected in sequence. The output module has a softmax activation function for classifying and predicting driving styles.

[0072] Optionally, the data with a dimension of 32×600×18 is input into the MAK model using the input module, and then passes through the Mamba module, the multi-head attention mechanism module, the feature fusion module, the global average pooling module, the KAN network module and the output module in sequence to obtain data with dimensions of 32×600×18, 32×600×18, 32×600×32, 32×36, 32×3 and 32×1, respectively. The data dimensions can be uniformly expressed as a×b×c and a×c, where a represents the number of samples in the data, b represents the time series length of the samples, and c represents the number of features.

[0073] Reference Manual Attached Figure 3 , showing the structural intent of the Mamba module provided in an embodiment of the present invention.

[0074] like Figure 3 The Mamba module includes a first channel and a second channel, wherein the first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit and a second projection unit connected in sequence, and the second channel includes a third projection unit and a second activation function unit, wherein the second channel is connected to the channel fusion unit, and the first channel and the second channel have the same input data.

[0075] S3: Construct a MAK model, wherein the MAK model includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module and an output module connected in sequence, the Mamba module includes a first channel and a second channel, wherein the first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit and a second projection unit connected in sequence, the second channel includes a third projection unit and a second activation function unit, wherein the second channel is connected to the channel fusion unit, and the first channel and the second channel have the same input data.

[0076] It should be noted that by constructing a MAK model composed of multiple modules, it is possible to efficiently process complex multi-source sensor data, automatically extract key features and perform multi-level feature fusion. Innovative technologies such as multi-head attention mechanism and channel fusion enable the model to better capture the spatiotemporal correlation in driving data and ensure the accuracy and stability of driving style recognition. The combination of the global average pooling module and the KAN network module simplifies feature processing, ensures the computational efficiency of the model, and helps the model achieve real-time prediction in resource-limited environments.

[0077] S4: Input the training set into the MAK model to train the MAK model.

[0078] It should be noted that by inputting the training set into the MAK model for training, the model can learn the complex relationship between driving style and sensor data, automatically adjust parameters, and improve the recognition accuracy of driving style, which helps the model gradually master the pattern characteristics, so that it can accurately predict driving behavior in different driving scenarios, and ultimately achieve efficient and automated driving style recognition.

[0079] S5: Get real-time driving records.

[0080] Among them, real-time driving records refer to multi-source sensor data collected in real time during vehicle driving, including but not limited to GPS signals, inertial measurement data (such as acceleration, angular velocity), vehicle sensor data (such as speed, braking, steering angle, etc.), which reflect the current dynamic behavior and driving conditions of the vehicle.

[0081] It should be noted that by obtaining real-time driving records, the model can be provided with the latest and real-time sensor data, so that the driving style recognition model can make predictions based on the current driving conditions. The application of real-time data ensures that the model can dynamically adapt to changes in driver behavior and provide timely feedback on driving style, thereby improving driving safety and the timeliness of user experience.

[0082] S6: Input the real-time driving record into the trained MAK model and output the predicted driving style.

[0083] In a possible implementation, S6 specifically includes:

[0084] S601: inputting the real-time driving record into the Mamba module through the input module for fusion, and extracting the short-term spatiotemporal features and long-term spatiotemporal features of the driving style;

[0085] Among them, short-term spatiotemporal characteristics represent the dynamic characteristics of the vehicle in a shorter period of time, such as sudden acceleration, sudden braking, sudden turning, etc., reflecting the driver's instantaneous behavior. Long-term spatiotemporal characteristics represent the driving mode and behavior changes of the vehicle in a longer period of time, such as continuous speed changes or long-term driving trajectories, reflecting the driver's overall style and habits.

[0086] In a possible implementation, S601 specifically includes:

[0087] S601A: Project the real-time driving record into a high-dimensional space using a first projection unit to obtain first driving projection data.

[0088] Specifically, the dimension of the driving sequence data is changed from (B, L, D) to (B, L, 2D) to obtain the projected driving sequence data, where B represents the number of implemented driving data, L represents the sequence length of the implemented driving data, and D represents the feature dimension of the implemented driving data.

[0089] It should be noted that by projecting real-time driving data into a high-dimensional space, it is possible to better capture hidden complex features, reveal the potential patterns of the data, and provide a basis for subsequent models to extract deeper driving style features.

[0090] S601B: Perform convolution and activation processing on the first driving projection data through a local convolution unit and an activation function unit to obtain a first output feature:

[0091] X conv =SiLU(W conv *X p )

[0092] Among them, X conv represents the first output feature, SiLU represents the SiLU activation function, and W conv represents the convolution kernel, K represents the size of the convolution kernel, X p represents the first driving projection data and * represents the convolution operation.

[0093] Specifically, by performing convolution and nonlinear activation processing on the first driving projection data through the local convolution unit and the activation function unit, local features can be effectively extracted, the model's ability to capture complex driving behavior patterns can be enhanced, and the model's nonlinear expression ability can be improved, which helps to more accurately identify driving styles.

[0094] S601C: gradually update the hidden state and output of the first output feature at each moment through the selective state space unit to obtain the second output feature:

[0095] X ssm =S6(X conv )

[0096] Among them, X ssm represents the second feature, and S6 represents the selective state space unit function.

[0097] It should be noted that by gradually updating the hidden state and output at each moment through the selective state space unit, the time series dependency of driving data can be effectively captured, important timing information can be retained, and the model can be ensured to more accurately identify dynamic changes in driving behavior, thereby improving recognition accuracy.

[0098] S601D: Project the real-time driving record into a high-dimensional space using a third projection unit to obtain second driving projection data.

[0099] S601E: Activate the second driving projection data through the second activation function unit to obtain the third output feature:

[0100] X act =SiLU(Xp )

[0101] Among them, X act represents the third output feature.

[0102] S601F: The second output feature and the third output feature are fused through the channel fusion unit to obtain the fused feature:

[0103] X fuse =X act ×X ssm

[0104] Among them, X fuse Represents fusion features.

[0105] It should be noted that by fusing the second output feature and the third output feature through the channel fusion unit, it is possible to comprehensively utilize the feature information of different channels, improve the richness and accuracy of feature expression, enhance the model's ability to process multi-dimensional data, and thus improve the overall performance of driving style recognition.

[0106] S601G: Using the second projection unit, perform a back-projection operation on the fused features to obtain short-term spatiotemporal features and long-term spatiotemporal features of the driving style.

[0107] It should be noted that by passing real-time data into the Mamba module through the input module, the model can effectively integrate short-term and long-term spatiotemporal features, thereby accurately capturing the driver's instantaneous behavior and long-term driving habits, improving the model's comprehensive understanding of driving style, making predictions more accurate, and improving the stability and response speed of the system.

[0108] S602: Use the multi-head attention mechanism module to weight the short-term spatiotemporal features and the long-term spatiotemporal features to obtain the attention features:

[0109]

[0110] X attention =MultiHeadAttention(Q,K,V)=Concat(head1,head2,head3)W O

[0111] head i =Attention(QW i Q ,KW i K ,VW i V )

[0112] Among them, Q represents the query vector, K represents the key vector, V represents the value vector, Attention represents the attention mechanism, softmax represents the softmax activation function, T represents the transposition operation, and d k represents the size of the key vector feature dimension, D represents the total feature dimension, H represents the number of heads of multi-head attention, and X attention represents the output features of the multi-head attention mechanism, MultiHeadAttention represents the multi-head attention mechanism, Concat represents the concatenation operation, head1, head2 and head3 represent the calculation results of the first attention head, the second attention head and the third attention head respectively, and W i Q , W i K and W i V Respectively represent the i-th attention head i The linear projection matrix corresponding to the query matrix, key matrix, and value matrix.

[0113] Among them, weight allocation means that when the model processes data, different weights are assigned to different features according to their importance. The larger the weight, the more critical the model believes the feature is to the result.

[0114] It should be noted that by assigning weights to short-term and long-term spatiotemporal features through the multi-head attention mechanism, the model can automatically identify the most important features for driving style recognition, thereby improving the accuracy of recognition. Different attention heads can focus on features on different time scales at the same time, ensuring that key driving behaviors are not ignored.

[0115] S603: Fusing short-term spatiotemporal features, long-term spatiotemporal features, and attention features to obtain a second fusion feature:

[0116]

[0117] Among them, X fusion represents the second fusion feature, cat represents the concatenation operation, and X mamba Represents the attention feature, dim=-1 means the fusion dimension is the last dimension.

[0118] S604: Reduce the dimension of the fused features through the global average pooling module and convert them into low-dimensional feature vectors:

[0119] X pool =GlobalAvgPool(X fusion )

[0120] Among them, X pool Represents a low-dimensional feature vector, and GlobalAvgPool represents global average pooling.

[0121] It should be noted that the global average pooling module is used to reduce the dimensionality of high-dimensional features, simplifying the data structure and reducing the computational complexity, while retaining global information and improving the efficiency and stability of the model.

[0122] S605: Classify low-dimensional feature vectors using the KAN network:

[0123]

[0124] Φ i ={φ q,p}

[0125] Where KAN represents the KAN network, Φ1 and Φ2 represent the first transformation function and the second transformation function for extracting different levels of features of the low-dimensional feature vector, respectively. i The i-th transformation function, i=1 represents the first transformation function, i=2 represents the second transformation function, represents the function transformation operation, φ q,p represents a univariate function with respect to the pth input dimension and the qth output dimension, p = 1, 2, ... n in , q=1,2…n out , n in Indicates the maximum input dimension, n out Indicates the maximum output dimension.

[0126] It should be noted that using the KAN network to classify low-dimensional feature vectors can effectively capture the complex patterns in the features, improve classification accuracy, and ensure that the model can accurately identify different driving styles.

[0127] S606: Based on the classified low-dimensional feature vector, the output module is used to calculate the probability of each driving style and generate a driving style prediction result:

[0128] y pre =SoftMax(KAN(x pool ))

[0129] Among them, SoftMax represents the SoftMax activation function, y pre Represents the driving style prediction result.

[0130] It should be noted that by calculating the probability of each driving style through the output module, the model can quantify the possibility of different driving behaviors, provide clear prediction results, and improve the interpretability and reliability of driving style recognition.

[0131] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0132] In the present invention, features are automatically extracted from multi-source sensor data through the MAK model without relying on manual feature extraction, thereby improving the efficiency of feature extraction, and being able to discover potential and complex features, thereby improving the accuracy of recognition. The KAN network module and the global average pooling module can extract stable and global features in long time series, which can better reflect the driver's personality and driving habits, and reduce the impact of short-term behavior fluctuations on recognition results. Through the efficient calculation of the convolutional network unit, the KAN model can run quickly in a resource-constrained environment, ensuring the real-time performance of the model. Through the dual-channel structure of the Mamba module, the model can extract features from multiple angles and integrate various types of information through channel fusion, thereby improving the stability of the model, reducing the dependence on specific data sets, and being able to accurately identify a variety of complex driving scenarios.

[0133] Reference Manual Attached Figure 4 , showing a structural schematic diagram of an end-to-end driving style recognition system based on multi-source sensor data provided by the present invention.

[0134] The present invention further provides an end-to-end driving style recognition system 20 based on multi-source sensor data, which is applied to the above-mentioned end-to-end driving style recognition method based on multi-source sensor data, comprising:

[0135] Processor 201.

[0136] The memory 202 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 201 , an end-to-end driving style recognition method based on multi-source sensor data as in the method embodiment is implemented.

[0137] The end-to-end driving style recognition system 20 based on multi-source sensor data provided by the present invention can execute the above-mentioned end-to-end driving style recognition method based on multi-source sensor data and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0138] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0139] In the present invention, features are automatically extracted from multi-source sensor data through the MAK model without relying on manual feature extraction, thereby improving the efficiency of feature extraction, and being able to discover potential and complex features, thereby improving the accuracy of recognition. The KAN network module and the global average pooling module can extract stable and global features in long time series, which can better reflect the driver's personality and driving habits, and reduce the impact of short-term behavior fluctuations on recognition results. Through the efficient calculation of the convolutional network unit, the KAN model can run quickly in a resource-constrained environment, ensuring the real-time performance of the model. Through the dual-channel structure of the Mamba module, the model can extract features from multiple angles and integrate various types of information through channel fusion, thereby improving the stability of the model, reducing the dependence on specific data sets, and being able to accurately identify a variety of complex driving scenarios.

[0140] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0141] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0142] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0143] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0144] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0145] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0146] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0147] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0148] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0149] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0151] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0152] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, an end-to-end driving style recognition method based on multi-source sensor data as in the method embodiment is implemented.

[0153] A computer-readable storage medium provided by the present invention can implement the steps and effects of the end-to-end driving style recognition method based on multi-source sensor data of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0154] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0155] In the present invention, features are automatically extracted from multi-source sensor data through the MAK model without relying on manual feature extraction, thereby improving the efficiency of feature extraction, and being able to discover potential and complex features, thereby improving the accuracy of recognition. The KAN network module and the global average pooling module can extract stable and global features in long time series, which can better reflect the driver's personality and driving habits, and reduce the impact of short-term behavior fluctuations on recognition results. Through the efficient calculation of the convolutional network unit, the KAN model can run quickly in a resource-constrained environment, ensuring the real-time performance of the model. Through the dual-channel structure of the Mamba module, the model can extract features from multiple angles and integrate various types of information through channel fusion, thereby improving the stability of the model, reducing the dependence on specific data sets, and being able to accurately identify a variety of complex driving scenarios.

[0156] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0157] There are a few points to note:

[0158] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0159] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0160] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0161] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An end-to-end driving style recognition method based on multi-source sensor data, characterized in that: include: S1: Acquire historical driving records and driving styles corresponding to the historical driving records, wherein the historical driving records and the driving styles constitute a data set, wherein the historical driving records include GPS signal data, inertial measurement signal data, and vehicle detection data; S2: performing preprocessing including data synchronization, feature selection, data standardization and data segmentation on the data set to obtain a training set, a validation set and a test set of preset proportions; S3: Construct a MAK model, wherein the MAK model includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module and an output module connected in sequence, the Mamba module includes a first channel and a second channel, wherein the first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit and a second projection unit connected in sequence, the second channel includes a third projection unit and a second activation function unit, wherein the second channel is connected to the channel fusion unit, and the first channel and the second channel have the same input data; S4: inputting the training set into the MAK model to train the MAK model; S5: Obtain real-time driving records; S6: Input the real-time driving record into the trained MAK model and output the predicted driving style.

2. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that: The driving styles include normal driving, slow driving and aggressive driving, and S1 is specifically: The data set is acquired using sensors built into the mobile phone.

3. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that: The data synchronization specifically includes: The frequency of collecting the GPS signal data and the vehicle measurement data is used as a sampling frequency; Selecting the highest sampling frequency among the sampling frequencies as the target frequency; Extracting sampling timestamps from the GPS signal data and the vehicle detection data respectively; According to the target frequency and the sampling timestamp, the GPS signal data is interpolated using a linear interpolation function, and the vehicle detection data is interpolated using a nearest neighbor interpolation function, so as to synchronize the GPS signal data and the vehicle detection data to the same time series: y(t)=y i Among them, x(t) represents the interpolation result at time t, x i represents the i-th known data point collected, x i+1 represents the i+1th known data point collected, t i Represents a known data point x i The corresponding time, t i+1 Represents a known data point x i+1 The corresponding time, t i <t<t i+1 , y(t) represents the nearest neighbor interpolation at time t, y i Represents the discrete data value at time t calculated by the nearest neighbor interpolation function.

4. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that: The feature selection is specifically as follows: Selecting features in the GPS signal data, the inertial measurement signal data, and the vehicle detection data as a feature set, wherein the features in the GPS signal data include vehicle speed and vehicle heading change, and the features in the inertial measurement signal data include acceleration of each axis of the sensor, jerk of each axis of the sensor, sensor flip angle, sensor pitch angle, sensor yaw angle, and acceleration of each axis of the sensor after Kalman filtering; The jerk of each axis of the sensor is calculated based on the acceleration of each axis in the inertial measurement signal data: Among them, j t represents the jerk at time t, a t represents the acceleration at time t, a t-1 represents the acceleration at time t-1, and dt represents the time difference between time t and time t-1; The features in the vehicle detection data include the distance to the preceding vehicle on the current lane, the time of collision with the preceding vehicle, the number of vehicles detected in the vehicle detection data, and the road type.

5. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that: The data standardization is specifically as follows: The feature set is standardized to have a mean value of 0 and a standard deviation of 1: Among them, z t represents the standardized value at time t, x t represents the feature data value at time t, μ represents the mean value of the feature vector, and σ represents the standard deviation of the feature vector.

6. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that: The data segmentation specifically includes: According to the time sequence, the standardized feature set is segmented using a sliding time window to obtain training sets, validation sets, and test sets with preset proportions.

7. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that: The S6 specifically includes: S601: inputting the real-time driving record into the Mamba module through the input module for fusion, and extracting the short-term spatiotemporal features and long-term spatiotemporal features of the driving style; S602: Use the multi-head attention mechanism module to weight the short-term spatiotemporal features and the long-term spatiotemporal features to obtain attention features: X attention =MultiHeadAttention(Q,K,V)=Concat(head1,head2,head3)W O head i =Attention(QW i Q ,KW i K ,VW i V ) Among them, Q represents the query vector, K represents the key vector, V represents the value vector, Attention represents the attention mechanism, softmax represents the softmax activation function, T represents the transposition operation, and d k represents the size of the key vector feature dimension, D represents the total feature dimension, H represents the number of heads of multi-head attention, and X attention represents the output features of the multi-head attention mechanism, MultiHeadAttention represents the multi-head attention mechanism, Concat represents the concatenation operation, head1, head2 and head3 represent the calculation results of the first attention head, the second attention head and the third attention head respectively, and W i Q , W i K and W i V The i-th attention head i The linear projection matrix corresponding to the query matrix, key matrix and value matrix; S603: Fusing the short-term spatiotemporal features, the long-term spatiotemporal features and the attention features to obtain a second fused feature: Among them, X fusion represents the second fusion feature, cat represents the concatenation operation, and X mamba Represents the attention feature, dim=-1 means the fusion dimension is the last dimension; S604: Reduce the dimension of the fused feature through the global average pooling module and convert it into a low-dimensional feature vector: X pool =GlobalAvgPool(X fusion ) Among them, X pool Represents a low-dimensional feature vector, GlobalAvgPool represents global average pooling; S605: Classify the low-dimensional feature vector using the KAN network: F i ={φ q,p } Where KAN represents the KAN network, Φ1 and Φ2 represent the first transformation function and the second transformation function for extracting different levels of features of the low-dimensional feature vector, respectively. i The i-th transformation function, i=1 represents the first transformation function, i=2 represents the second transformation function, represents the function transformation operation, φ q,p represents a univariate function with respect to the pth input dimension and the qth output dimension, p = 1, 2, ... n in , q=1,2…n out , n in Indicates the maximum input dimension, n out Indicates the maximum output dimension; S606: Based on the classified low-dimensional feature vector, the output module is used to calculate the probability of each driving style and generate a driving style prediction result: y pre =SoftMax(CAN(x pool )) Among them, SoftMax represents the SoftMax activation function, y pre Represents the driving style prediction result.

8. The end-to-end driving style recognition method based on multi-source sensor data according to claim 7, characterized in that: The S601 specifically includes: S601A: Projecting the real-time driving record into a high-dimensional space using the first projection unit to obtain first driving projection data; S601B: Perform convolution and activation processing on the first driving projection data through the local convolution unit and the activation function unit to obtain a first output feature: X conv =SiLU(W conv *X p ) Among them, X conv represents the first output feature, SiLU represents the SiLU activation function, and W conv represents the convolution kernel, X p represents the first driving projection data, * represents the convolution operation; S601C: gradually updating the hidden state and output of the first output feature at each moment through the selective state space unit to obtain the second output feature: X ssm =S6(X conv ) Among them, X ssm represents the second feature, S6 represents the selective state space unit function; S601D: Projecting the real-time driving record into a high-dimensional space using the third projection unit to obtain second driving projection data; S601E: Activate the second driving projection data through the second activation function unit to obtain a third output feature: X act =SiLU(X p ) Among them, X act represents the third output feature; S601F: The second output feature and the third output feature are fused by the channel fusion unit to obtain a fused feature: X fuse =X act ×X ssm Among them, X fuse Indicates fusion features; S601G: Utilize the second projection unit to perform a back-projection operation on the fused features to obtain short-term spatiotemporal features and long-term spatiotemporal features of the driving style.

9. An end-to-end driving style recognition system based on multi-source sensor data, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the end-to-end driving style recognition method based on multi-source sensor data as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the end-to-end driving style recognition method based on multi-source sensor data as described in any one of claims 1 to 8 is implemented.

Images