An end-to-end driving style recognition method and system based on multi-source sensor data

Abstract

This invention provides an end-to-end driving style recognition method and system based on multi-source sensor data, belonging to the field of driving style recognition technology. The method includes: acquiring historical driving records and corresponding driving styles; performing preprocessing on the dataset, including data synchronization, feature selection, data standardization, and data segmentation, to obtain a training set, validation set, and test set with preset proportions; constructing a MAK model; inputting the training set into the MAK model to train the MAK model; acquiring real-time driving records; inputting the real-time driving records into the trained MAK model, and outputting a predicted driving style. This invention can accurately capture the complex spatiotemporal features of driving behavior, achieving efficient and real-time driving style recognition.

Description

Technical Field

[0001] This invention relates to the field of driving style recognition technology, and in particular to an end-to-end driving style recognition method and system based on multi-source sensor data. Background Technology

[0002] End-to-end driving style recognition methods based on multi-source sensor data are a deep learning technique that combines data from multiple sensors (such as accelerometer, gyroscope, GPS, etc.) to automatically identify a driver's driving style. This method learns directly from sensor data and generates driving behavior classification results without the need for complex manual feature extraction. Through this type of method, driving safety can be improved, user experience can be enhanced, and applications of autonomous driving and driving behavior analysis can be supported.

[0003] With the development of artificial intelligence, sensor technology, internet technology, big data, and 5G networks, intelligent vehicles based on electrification, intelligence, and connectivity have become a trend in the automotive industry. A potential but clear direction for intelligent vehicles is shifting from the initial focus on faster and more fuel-efficient vehicles to a more human-centered and personalized approach. Driving style is a summary of a driver's driving behavior and habits, reflecting the driver's driving characteristics. It has positive significance in areas such as driving safety, vehicle intelligence, vehicle energy efficiency, and the automotive insurance industry. Accurately identifying a driver's driving style is a highly valuable research endeavor.

[0004] However, traditional driving style recognition methods suffer from insufficient global feature extraction capabilities and poor performance with small samples when processing multivariate long-term series data. They are unable to capture complex driving behaviors, are inefficient, and are difficult to adapt to diverse driving scenarios. Using short time windows to identify driving styles makes them susceptible to the influence of instantaneous behavior and environmental changes, resulting in unstable recognition results that fail to accurately reflect the driver's long-term driving habits and personality. Summary of the Invention

[0005] To address the shortcomings of existing technologies, traditional driving style recognition methods typically rely on manual feature extraction, which is time-consuming and tedious, often overlooking hidden and useful features, failing to capture complex driving behaviors, being inefficient, and unable to adapt to diverse driving scenarios. Furthermore, using short time windows for driving style recognition makes them susceptible to instantaneous behavior and environmental changes, resulting in unstable recognition results that fail to accurately reflect a driver's long-term driving habits and personality. This invention provides an end-to-end driving style recognition method and system based on multi-source sensor data.

[0006] The technical solutions provided by the embodiments of the present invention are as follows:

[0007] First aspect

[0008] An end-to-end driving style recognition method based on multi-source sensor data provided in this invention includes:

[0009] S1: Obtain historical driving records and corresponding driving styles. The historical driving records and driving styles constitute a dataset. The historical driving records include GPS signal data, inertial measurement signal data, and vehicle detection data.

[0010] S2: Perform preprocessing on the dataset, including data synchronization, feature selection, data standardization, and data segmentation, to obtain a training set, validation set, and test set with a preset ratio;

[0011] S3: Construct a MAK model, which includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module, and an output module connected in sequence. The Mamba module includes a first channel and a second channel. The first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit, and a second projection unit connected in sequence. The second channel includes a third projection unit and a second activation function unit. The second channel is connected to the channel fusion unit. The first channel and the second channel have the same input data.

[0012] S4: Input the training set into the MAK model to train the MAK model;

[0013] S5: Obtain real-time driving records;

[0014] S6: Input the real-time driving record into the trained MAK model and output the predicted driving style.

[0015] Second aspect

[0016] An end-to-end driving style recognition system based on multi-source sensor data provided in this embodiment of the invention includes:

[0017] processor;

[0018] The memory stores computer-readable instructions, which, when executed by a processor, implement the end-to-end driving style recognition method based on multi-source sensor data as described in the first aspect.

[0019] Third aspect

[0020] The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the end-to-end driving style recognition method based on multi-source sensor data as described in the first aspect.

[0021] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0022] In this invention, the MAK model automatically extracts features from multi-source sensor data, eliminating the need for manual feature extraction and improving its efficiency. It also discovers potential and complex features, enhancing recognition accuracy. The KAN network module and global average pooling module extract stable, global features from long-term sequences, better reflecting driver personality and driving habits and reducing the impact of short-term behavioral fluctuations on recognition results. Through the efficient computation of convolutional network units, the KAN model can run quickly in resource-constrained environments, ensuring real-time performance. The dual-channel structure of the Mamba module allows the model to extract features from multiple angles and integrate various types of information through channel fusion, thereby improving model stability, reducing dependence on specific datasets, and accurately recognizing various complex driving scenarios. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating an end-to-end driving style recognition method based on multi-source sensor data, provided in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the structure of the MAK model provided in an embodiment of the present invention;

[0026] Figure 3 The structural intent of the Mamba module provided in the embodiments of the present invention;

[0027] Figure 4 This is a schematic diagram of the structure of an end-to-end driving style recognition system based on multi-source sensor data, provided in an embodiment of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0029] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0030] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0031] Reference manual attached Figure 1 The diagram shows a flowchart of an end-to-end driving style recognition method based on multi-source sensor data provided by an embodiment of the present invention.

[0032] This invention provides an end-to-end driving style recognition method based on multi-source sensor data, the method comprising:

[0033] S1: Obtain historical driving records and corresponding driving styles. The historical driving records and driving styles constitute a dataset. The historical driving records include GPS signal data, inertial measurement signal data, and vehicle detection data.

[0034] Among them, historical driving records refer to data generated during driving over a long period of time through vehicle sensors, GPS, inertial measurement devices, etc. It is a driving record of multiple drivers performing various driving styles on highways and secondary roads. GPS signal data is vehicle position information collected by the Global Positioning System (GPS), including speed, direction and geographical location. Inertial measurement signal data is data collected by inertial measurement devices such as accelerometers and gyroscopes, reflecting the vehicle's acceleration, angular velocity and other motion information. Vehicle detection data is the sensor output information of the vehicle, such as vehicle speed, throttle, brake, steering angle, distance to the vehicle in front, etc.

[0035] In one possible implementation, driving styles include normal driving, sluggish driving, and aggressive driving, and S1 specifically refers to:

[0036] Data sets are acquired using the phone's built-in sensors.

[0037] Among them, the sensors built into the phone refer to the sensor components commonly found in modern smartphones, which can capture the vehicle's motion status and environmental information. Common built-in sensors in mobile phones include accelerometers, which measure the linear acceleration of the phone along different axes to reflect the vehicle's acceleration and deceleration; gyroscopes, which measure the phone's rotational angular velocity to reflect the vehicle's turning and rotational behavior; GPS modules, which collect location information; and magnetometers, which measure the Earth's magnetic field.

[0038] It should be noted that by acquiring rich historical driving records and multi-source sensor data (GPS, inertial measurement, vehicle sensors, etc.), it is possible to comprehensively reflect the driver's driving behavior and provide a high-quality dataset with real driving style labels for the driving style recognition model.

[0039] S2: Perform preprocessing on the dataset, including data synchronization, feature selection, data standardization, and data segmentation, to obtain a training set, validation set, and test set with a preset ratio.

[0040] Data synchronization aligns data from different sensors to a unified time base. Since different sensors may have different sampling frequencies, data synchronization ensures data consistency at the same point in time. Feature selection extracts key features useful for driving style recognition from the dataset. Data standardization scales data with different features to the same scale. Data segmentation divides the preprocessed dataset into training, validation, and test sets. The training set is a portion of the dataset specifically used to train the machine learning model, which learns characteristic patterns of driving style on this data. The validation set monitors model performance during training, helping to adjust hyperparameters to prevent overfitting or underfitting. The test set is used for final evaluation after model training to check the model's performance on unseen data.

[0041] In one possible implementation, data synchronization specifically includes:

[0042] The frequency at which GPS signal data and vehicle measurement data are collected is used as the sampling frequency.

[0043] The sampling frequency is the rate at which data is acquired; typically, different sensors have different sampling frequencies.

[0044] It should be noted that by specifying the frequency of collecting GPS signals and vehicle measurement data, the time accuracy during the data collection process can be ensured, and sufficient details can be captured, especially in rapidly changing driving scenarios.

[0045] Select the highest sampling frequency among the sampling frequencies as the target frequency.

[0046] Extract the sampling timestamps from the GPS signal data and vehicle detection data respectively.

[0047] Among them, the sampling timestamp is a time stamp for each data point, recording the time when the data was collected at a specific point in time.

[0048] It should be noted that extracting the sampling timestamps from the GPS signal data and vehicle detection data separately can ensure the consistency of the data sequence, avoid analysis errors caused by time differences, and ensure the accuracy of identifying driving behavior.

[0049] Based on the target frequency and sampling timestamp, the GPS signal data is interpolated using a linear interpolation function, and the vehicle detection data is interpolated using a nearest neighbor interpolation function, synchronizing the GPS signal data and vehicle detection data to the same time series.

[0050]

[0051] y(t)=y i

[0052] Where x(t) represents the interpolation result at time t, x i Let x represent the i-th known data point collected. i+1 Let t represent the (i+1)th known data point collected. i Represents the known data point x i At the corresponding time, t i+1 Represents the known data point x i+1 At the corresponding time, t i <t<t i+1 y(t) represents the nearest neighbor interpolation at time t. i This represents the discrete data value at time t calculated using the nearest neighbor interpolation function.

[0053] In one possible implementation, feature selection specifically involves:

[0054] Features from GPS signal data, inertial measurement signal data, and vehicle detection data are selected as the feature set. The features in the GPS signal data include vehicle speed and vehicle heading changes, while the features in the inertial measurement signal data include the acceleration of each axis of the sensor, the jerk of each axis of the sensor, the flip angle, the pitch angle, the yaw angle, and the acceleration of each axis of the sensor after Kalman filtering.

[0055] Among these metrics, vehicle speed represents the vehicle's travel speed and is used to reflect acceleration and deceleration behaviors in driving style. Vehicle heading change is the change in steering of the vehicle during driving, usually expressed as an angle, and can reflect the driver's steering habits. Inertial measurement signal data is the vehicle motion state data collected by inertial measurement units (IMUs) such as accelerometers and gyroscopes, reflecting the vehicle's acceleration and rotational motion. The acceleration of each sensor axis is the linear acceleration along different axes (X, Y, Z) of the vehicle, used to measure the vehicle's acceleration, braking, and turning behaviors. The jerk of each sensor axis is the rate of change of acceleration, representing the sudden change in vehicle acceleration or deceleration, reflecting the abruptness of driving behavior. The sensor roll angle is the angle at which the vehicle rolls along its longitudinal axis, used to measure the vehicle's roll when turning. The sensor pitch angle is the angle at which the vehicle tilts up and down along its lateral axis, usually used to detect the vehicle's uphill and downhill conditions. The sensor yaw angle is the angle at which the vehicle rotates around its vertical axis, reflecting the vehicle's turning and heading changes. Kalman filtering is a signal processing technique used to remove noise from sensor data, thereby generating smoother and more accurate acceleration data.

[0056] It should be noted that by selecting key features from different sources, the driver's driving behavior patterns can be comprehensively and accurately reflected, further improving the accuracy and robustness of the data, reducing noise interference, providing high-quality input for the driving style recognition model, and improving recognition accuracy.

[0057] The jerk of each axis of the sensor is calculated based on the acceleration of each axis in the inertial measurement signal data:

[0058]

[0059] Where, j t a represents the jerk at time t. t a represents the acceleration at time t. t-1 Let dt represent the acceleration at time t-1, and dt represent the time difference between time t and time t-1.

[0060] Features in vehicle detection data include the distance to the vehicle in front in the current lane, the time of collision with the vehicle in front, the number of vehicles detected in the vehicle detection data, and the road type.

[0061] In one possible implementation, data standardization specifically involves:

[0062] Standardize the feature set to unify it into values ​​with a mean of 0 and a standard deviation of 1:

[0063]

[0064] Among them, z t x represents the standardized value at time t.t Let μ represent the characteristic data value at time t, μ represent the average value of the eigenvector, and σ represent the standard deviation of the eigenvector.

[0065] The mean refers to the central value of the dataset, representing the central tendency of the data points. In the standardization process, the mean of the data points is set to 0 to eliminate bias. The standard deviation is the dispersion of the dataset, representing the degree of deviation of the data points from the mean. In the standardization process, the standard deviation is set to 1 to eliminate the dimensional differences between features and make their distribution consistent.

[0066] It should be noted that by standardizing the feature set, the scale difference between different features is eliminated, making the influence of features on the model more balanced. This helps to improve the convergence speed of the model, avoid the excessive influence of certain features with large dimensions on the model, and improve the prediction accuracy and stability of the model.

[0067] In one possible implementation, data segmentation specifically includes:

[0068] Based on the time sequence, the standardized feature set is segmented using a sliding time window to obtain a training set, validation set, and test set with a preset ratio.

[0069] It should be noted that by using data synchronization, feature selection, data standardization, and data segmentation, data quality is ensured and model performance is optimized. Data synchronization eliminates the differences in sampling time between different sensors, feature selection improves recognition efficiency, standardization reduces the influence of units, and data segmentation ensures the model's generalization ability, making it perform stably on new data.

[0070] Reference manual attached Figure 2 The diagram shows a schematic representation of the MAK model provided in an embodiment of the present invention.

[0071] like Figure 2 The MAK model consists of an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module, and an output module, all connected in sequence. The output module contains a softmax activation function for classifying and predicting driving styles.

[0072] Optionally, the input module inputs data with a dimension of 32×600×18 into the MAK model, and then sequentially passes through the Mamba module, multi-head attention mechanism module, feature fusion module, global average pooling module, KAN network module, and output module to obtain data with dimensions of 32×600×18, 32×600×18, 32×600×32, 32×36, 32×3, and 32×1, respectively. Here, the data dimension can be uniformly represented as a×b×c and a×c, where a represents the number of samples in the data, b represents the time series length of the samples, and c represents the number of features.

[0073] Reference manual attached Figure 3 This illustrates the structural intent of the Mamba module provided in an embodiment of the present invention.

[0074] like Figure 3 The Mamba module includes a first channel and a second channel. The first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit, and a second projection unit connected in sequence. The second channel includes a third projection unit and a second activation function unit. The second channel is connected to the channel fusion unit. The first channel and the second channel have the same input data.

[0075] S3: Construct a MAK model, which includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module, and an output module connected in sequence. The Mamba module includes a first channel and a second channel. The first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit, and a second projection unit connected in sequence. The second channel includes a third projection unit and a second activation function unit. The second channel is connected to the channel fusion unit. The first channel and the second channel have the same input data.

[0076] It should be noted that by constructing a MAK model composed of multiple modules, it can efficiently process complex multi-source sensor data, automatically extract key features and perform multi-level feature fusion. Innovative technologies such as multi-head attention mechanism and channel fusion enable the model to better capture the spatiotemporal correlation in driving data, ensuring the accuracy and stability of driving style recognition. The combination of global average pooling module and KAN network module simplifies feature processing, ensures the computational efficiency of the model, and helps the model achieve real-time prediction in resource-constrained environments.

[0077] S4: Input the training set into the MAK model to train the MAK model.

[0078] It should be noted that by inputting the training set into the MAK model for training, the model can learn the complex relationship between driving style and sensor data, automatically adjust parameters, improve the accuracy of driving style recognition, and help the model gradually master pattern features, enabling it to accurately predict driving behavior in different driving scenarios, and ultimately achieve efficient and automated driving style recognition.

[0079] S5: Get real-time driving records.

[0080] Real-time driving records refer to multi-source sensor data collected in real time during vehicle operation, including but not limited to GPS signals, inertial measurement data (such as acceleration and angular velocity), and vehicle sensor data (such as speed, braking, and steering angle). These data reflect the current dynamic behavior and driving conditions of the vehicle.

[0081] It should be noted that by acquiring real-time driving records, the model can be provided with the latest and most up-to-date sensor data, enabling the driving style recognition model to make predictions based on the current driving situation. The application of real-time data ensures that the model can dynamically adapt to changes in driver behavior and provide timely feedback on driving style, thereby improving driving safety and the timeliness of user experience.

[0082] S6: Input the real-time driving record into the trained MAK model and output the predicted driving style.

[0083] In one possible implementation, S6 specifically includes:

[0084] S601: The real-time driving record is input into the Mamba module through the input module for fusion, and short-term and long-term spatiotemporal features of driving style are extracted;

[0085] Among them, short-term spatiotemporal characteristics represent the dynamic characteristics of a vehicle within a short period of time, such as rapid acceleration, sudden braking, and sudden steering, reflecting the driver's instantaneous behavior. Long-term spatiotemporal characteristics represent the changes in driving patterns and behaviors of a vehicle within a longer period of time, such as continuous speed changes or long-term driving trajectories, reflecting the driver's overall style and habits.

[0086] In one possible implementation, S601 specifically includes:

[0087] S601A: The real-time driving record is projected onto a high-dimensional space using the first projection unit to obtain the first driving projection data.

[0088] Specifically, the dimension of the driving sequence data is changed from (B,L,D) to (B,L,2D) to obtain the projected driving sequence data, where B represents the number of driving data implementations, L represents the sequence length of the driving data implementations, and D represents the feature dimension of the driving data implementations.

[0089] It should be noted that by projecting real-time driving data into a high-dimensional space, we can better capture hidden complex features, reveal potential patterns in the data, and provide a foundation for subsequent models to extract deeper driving style features.

[0090] S601B: The first output feature is obtained by performing convolution and activation processing on the first driving projection data through local convolution units and activation function units.

[0091] X conv =SiLU(W conv *X p )

[0092] Among them, X conv Let W represent the first output feature, SiLU represent the SiLU activation function, and W represent the first output feature. conv X represents the convolution kernel, K represents the size of the convolution kernel, and X represents the size of the convolution kernel. p This represents the first driver's projection data, and * indicates a convolution operation.

[0093] Specifically, by performing convolution and nonlinear activation processing on the first driving projection data through local convolution units and activation function units, local features can be effectively extracted, enhancing the model's ability to capture complex driving behavior patterns and improving the model's nonlinear expression ability, which helps to identify driving styles more accurately.

[0094] S601C: The second output feature is obtained by progressively updating the hidden state and output of the first output feature at each time step using selective state-space units.

[0095] X ssm =S6(X conv )

[0096] Among them, X ssm S6 represents the second characteristic and the selective state-space unit function.

[0097] It should be noted that by selectively updating the hidden state and output at each time step by step using state-space units, the temporal dependencies of driving data can be effectively captured, important temporal information can be preserved, and the model can be more accurately identified in the dynamic changes in driving behavior, thereby improving the recognition accuracy.

[0098] S601D: The real-time driving record is projected into a high-dimensional space using the third projection unit to obtain the second driving projection data.

[0099] S601E: The second activation function unit performs an activation operation on the two driver projection data to obtain the third output feature:

[0100] X act =SiLU(X)p )

[0101] Among them, X act This represents the third output feature.

[0102] S601F: The second and third output features are fused through the channel fusion unit to obtain the fused feature:

[0103] X fuse =X act ×X ssm

[0104] Among them, X fuse This indicates the fusion feature.

[0105] It should be noted that by fusing the second and third output features through the channel fusion unit, the feature information of different channels can be comprehensively utilized, improving the richness and accuracy of feature representation, enhancing the model's ability to process multidimensional data, and thus improving the overall performance of driving style recognition.

[0106] S601G: Using the second projection unit, back-projection is performed on the fused features to obtain short-term and long-term spatiotemporal features of driving style.

[0107] It should be noted that by inputting real-time data into the Mamba module through the input module, the model can effectively integrate short-term and long-term spatiotemporal features, thereby accurately capturing the driver's instantaneous behavior and long-term driving habits, improving the model's comprehensive understanding of driving style, making predictions more accurate, and enhancing the system's stability and response speed.

[0108] S602: The multi-head attention mechanism module is used to assign weights to short-term and long-term spatiotemporal features to obtain attention features:

[0109]

[0110] X attention =MultiHeadAttention(Q,K,V)=Concat(head1,head2,head3)W O

[0111] head i =Attention(QW i Q ,KW i K VW i V )

[0112] Where Q represents the query vector, K represents the key vector, V represents the value vector, Attention represents the attention mechanism, softmax represents the softmax activation function, T represents the transpose operation, and d k The key vector feature dimension is represented by D, the total feature dimension is represented by H, and the number of heads in the multi-head attention mechanism is represented by X. attention This represents the output features of the multi-head attention mechanism. `MultiHeadAttention` represents the multi-head attention mechanism, `Concat` represents the concatenation operation, and `head1`, `head2`, and `head3` represent the computation results of the first, second, and third attention heads, respectively. W i Q W i K and W i V These represent the i-th attention head. i The linear projection matrix corresponding to the query matrix, key matrix, and value matrix.

[0113] Weighting refers to assigning different weights to different features based on their importance when the model processes data. The larger the weight, the more critical the model considers the feature to be for the result.

[0114] It should be noted that by assigning weights to short-term and long-term spatiotemporal features through a multi-head attention mechanism, the model can automatically identify the features most important for driving style recognition, thereby improving the accuracy of recognition. Different attention heads can simultaneously focus on features at different time scales, ensuring that key driving behaviors are not ignored.

[0115] S603: By fusing short-term spatiotemporal features, long-term spatiotemporal features, and attention features, a second fused feature is obtained:

[0116]

[0117] Among them, X fusion This represents the second fusion feature, cat represents the concatenation operation, and X represents the second fusion feature. mamba This represents the attention feature, and dim = -1 indicates that the fusion dimension is the last dimension.

[0118] S604: The fused features are reduced in dimensionality and transformed into low-dimensional feature vectors through a global average pooling module.

[0119] X pool =GlobalAvgPool(X fusion )

[0120] Among them, X pool This represents a low-dimensional feature vector, and GlobalAvgPool represents global average pooling.

[0121] It should be noted that by using the global average pooling module to reduce the dimensionality of high-dimensional features, the data structure is simplified, the computational complexity is reduced, and global information is preserved, thereby improving the efficiency and stability of the model.

[0122] S605: Classifying low-dimensional feature vectors using KAN networks:

[0123]

[0124] Φ i ={φ q,p}

[0125] Where KAN represents the KAN network, Φ1 and Φ2 represent the first and second transformation functions for extracting features at different levels of the low-dimensional feature vector, respectively. i The i-th transformation function, where i=1 represents the first transformation function and i=2 represents the second transformation function. φ represents the function transformation operation. q,p Let p be a univariate function with respect to the p-th input dimension and the q-th output dimension, where p = 1, 2, ..., n. in q = 1, 2, ..., n out n in Indicates the maximum dimension of the input, n out This indicates the maximum output dimension.

[0126] It should be noted that using KAN networks to classify low-dimensional feature vectors can effectively capture complex patterns in the features, improve classification accuracy, and ensure that the model can accurately identify different driving styles.

[0127] S606: Based on the classified low-dimensional feature vectors, the output module calculates the probability of each driving style and generates driving style prediction results.

[0128] y pre =SoftMax(KAN(x) pool ))

[0129] Where SoftMax represents the SoftMax activation function, y pre This indicates the prediction result for driving style.

[0130] It should be noted that by calculating the probability of each driving style through the output module, the model can quantify the likelihood of different driving behaviors, provide clear prediction results, and improve the interpretability and reliability of driving style recognition.

[0131] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0132] In this invention, the MAK model automatically extracts features from multi-source sensor data, eliminating the need for manual feature extraction and improving its efficiency. It also discovers potential and complex features, enhancing recognition accuracy. The KAN network module and global average pooling module extract stable, global features from long-term sequences, better reflecting driver personality and driving habits and reducing the impact of short-term behavioral fluctuations on recognition results. Through the efficient computation of convolutional network units, the KAN model can run quickly in resource-constrained environments, ensuring real-time performance. The dual-channel structure of the Mamba module allows the model to extract features from multiple angles and integrate various types of information through channel fusion, thereby improving model stability, reducing dependence on specific datasets, and accurately recognizing various complex driving scenarios.

[0133] Reference manual attached Figure 4 The diagram shows a schematic of the structure of an end-to-end driving style recognition system based on multi-source sensor data provided by the present invention.

[0134] The present invention also provides an end-to-end driving style recognition system 20 based on multi-source sensor data, applied to the above-mentioned end-to-end driving style recognition method based on multi-source sensor data, comprising:

[0135] Processor 201.

[0136] The memory 202 stores computer-readable instructions, which, when executed by the processor 201, implement the end-to-end driving style recognition method based on multi-source sensor data as described in the method embodiment.

[0137] The end-to-end driving style recognition system 20 based on multi-source sensor data provided by the present invention can execute the above-mentioned end-to-end driving style recognition method based on multi-source sensor data and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate further.

[0138] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0139] In this invention, the MAK model automatically extracts features from multi-source sensor data, eliminating the need for manual feature extraction and improving its efficiency. It also discovers potential and complex features, enhancing recognition accuracy. The KAN network module and global average pooling module extract stable, global features from long-term sequences, better reflecting driver personality and driving habits and reducing the impact of short-term behavioral fluctuations on recognition results. Through the efficient computation of convolutional network units, the KAN model can run quickly in resource-constrained environments, ensuring real-time performance. The dual-channel structure of the Mamba module allows the model to extract features from multiple angles and integrate various types of information through channel fusion, thereby improving model stability, reducing dependence on specific datasets, and accurately recognizing various complex driving scenarios.

[0140] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0141] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0142] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0143] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0144] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0145] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0146] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0147] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0148] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If the functionality is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the end-to-end driving style recognition method based on multi-source sensor data as described in the method embodiments.

[0153] The present invention provides a computer-readable storage medium that can implement the steps and effects of the end-to-end driving style recognition method based on multi-source sensor data in the above-described method embodiments. To avoid repetition, the present invention will not repeat the details.

[0154] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0155] In this invention, the MAK model automatically extracts features from multi-source sensor data, eliminating the need for manual feature extraction and improving its efficiency. It also discovers potential and complex features, enhancing recognition accuracy. The KAN network module and global average pooling module extract stable, global features from long-term sequences, better reflecting driver personality and driving habits and reducing the impact of short-term behavioral fluctuations on recognition results. Through the efficient computation of convolutional network units, the KAN model can run quickly in resource-constrained environments, ensuring real-time performance. The dual-channel structure of the Mamba module allows the model to extract features from multiple angles and integrate various types of information through channel fusion, thereby improving model stability, reducing dependence on specific datasets, and accurately recognizing various complex driving scenarios.

[0156] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0157] The following points need to be explained:

[0158] (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0159] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the present invention; that is, these drawings are not drawn to actual scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element, or there may be intermediate elements.

[0160] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0161] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An end-to-end driving style recognition method based on multi-source sensor data, characterized in that, include: S1: Obtain historical driving records and the driving styles corresponding to the historical driving records, wherein the historical driving records and the driving styles constitute a dataset, and the historical driving records include GPS signal data, inertial measurement signal data and vehicle detection data; S2: Perform preprocessing on the dataset, including data synchronization, feature selection, data standardization, and data segmentation, to obtain a training set, validation set, and test set with a preset ratio; S3: Construct a MAK model, wherein the MAK model includes an input module, a Mamba module, a multi-head attention mechanism module, a feature fusion module, a global average pooling module, a KAN network module, and an output module connected in sequence. The Mamba module includes a first channel and a second channel. The first channel includes a first projection unit, a local convolution unit, a first activation function unit, a selective state space unit, a channel fusion unit, and a second projection unit connected in sequence. The second channel includes a third projection unit and a second activation function unit. The second channel is connected to the channel fusion unit. The first channel and the second channel have the same input data. S4: Input the training set into the MAK model to train the MAK model; S5: Obtain real-time driving records; S6: Input the real-time driving record into the trained MAK model and output the predicted driving style.

2. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that, The driving styles include normal driving, sluggish driving, and aggressive driving, and S1 specifically refers to: The dataset is acquired using the phone's built-in sensors.

3. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that, The data synchronization specifically includes: The frequency at which the GPS signal data and the vehicle measurement data are collected is used as the sampling frequency; Select the highest sampling frequency among the sampling frequencies as the target frequency; Extract the sampling timestamps from the GPS signal data and the vehicle detection data respectively; Based on the target frequency and the sampling timestamp, the GPS signal data is interpolated using a linear interpolation function, and the vehicle detection data is interpolated using a nearest neighbor interpolation function, synchronizing the GPS signal data and vehicle detection data to the same time series. y(t)=y i Where x(t) represents the interpolation result at time t, x i Let x represent the i-th known data point collected. i+1 Let t represent the (i+1)th known data point collected. i Represents the known data point x i At the corresponding time, t i+1 Represents the known data point x i+1 At the corresponding time, t i <t<t i+1 y(t) represents the nearest neighbor interpolation at time t. i This represents the discrete data value at time t calculated using the nearest neighbor interpolation function.

4. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that, The feature selection specifically refers to: Features from the GPS signal data, the inertial measurement signal data, and the vehicle detection data are selected as a feature set. The features in the GPS signal data include vehicle speed and vehicle heading changes, and the features in the inertial measurement signal data include the acceleration of each axis of the sensor, the jerk of each axis of the sensor, the flip angle, the pitch angle, the yaw angle, and the acceleration of each axis of the sensor after Kalman filtering. The jerk of each axis of the sensor is calculated based on the acceleration of each axis in the inertial measurement signal data: Where, j t a represents the jerk at time t. t a represents the acceleration at time t. t-1 Let dt represent the acceleration at time t-1, and dt represent the time difference between time t and time t-1. The features in the vehicle detection data include the distance to the vehicle in front in the current lane, the time of collision with the vehicle in front, the number of vehicles detected in the vehicle detection data, and the road type.

5. The end-to-end driving style recognition method based on multi-source sensor data according to claim 4, characterized in that, The data standardization specifically refers to: The feature set is standardized to a set of values ​​with a mean of 0 and a standard deviation of 1. Among them, z t x represents the standardized value at time t. t Let μ represent the characteristic data value at time t, μ represent the mean of the eigenvector, and σ represent the standard deviation of the eigenvector.

6. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that, The data segmentation specifically includes: Based on the time sequence, the standardized feature set is segmented using a sliding time window to obtain a training set, validation set, and test set with a preset ratio.

7. The end-to-end driving style recognition method based on multi-source sensor data according to claim 1, characterized in that, S6 specifically includes: S601: The real-time driving record is input to the Mamba module through the input module for fusion, and the short-term and long-term spatiotemporal features of the driving style are extracted; S602: The multi-head attention mechanism module is used to assign weights to the short-term and long-term spatiotemporal features to obtain attention features: X attention =MultiHeadAttention(Q,K,V)=Concat(head1,head2,head3)W O head i =Attention(QW i Q ,KW i K ,VW i V ) Where Q represents the query vector, K represents the key vector, V represents the value vector, Attention represents the attention mechanism, softmax represents the softmax activation function, T represents the transpose operation, and d k The key vector feature dimension is represented by D, the total feature dimension is represented by H, and the number of heads in the multi-head attention mechanism is represented by X. attention This represents the output features of the multi-head attention mechanism. `MultiHeadAttention` represents the multi-head attention mechanism, `Concat` represents the concatenation operation, and `head1`, `head2`, and `head3` represent the computation results of the first, second, and third attention heads, respectively. W i Q W i K and W i V The i-th attention head i The linear projection matrix corresponding to the query matrix, key matrix, and value matrix; S603: By fusing the short-term spatiotemporal features, long-term spatiotemporal features, and attention features, a second fused feature is obtained: Among them, X fusion This represents the second fusion feature, cat represents the concatenation operation, and X represents the second fusion feature. mamba This represents the attention feature, where dim = -1 indicates that the fusion dimension is the last dimension. S604: The fused features are reduced in dimensionality and transformed into low-dimensional feature vectors through the global average pooling module. X pool =GlobalAvgPool(X fusion ) Among them, X pool Represents a low-dimensional feature vector; GlobalAvgPool represents global average pooling. S605: Classify the low-dimensional feature vector using the KAN network: F i ={φ q,p } Where KAN represents the KAN network, Φ1 and Φ2 represent the first and second transformation functions for extracting features at different levels of the low-dimensional feature vector, respectively. i The i-th transformation function, where i=1 represents the first transformation function and i=2 represents the second transformation function. φ represents the function transformation operation. q,p Let p be a univariate function with respect to the p-th input dimension and the q-th output dimension, where p = 1, 2, ..., n. in q = 1, 2, ..., n out n in Indicates the maximum dimension of the input, n out Indicates the maximum dimension of the output; S606: Based on the classified low-dimensional feature vectors, the output module calculates the probability of each driving style and generates driving style prediction results. y pre =SoftMax(CAN(x pool )) Where SoftMax represents the SoftMax activation function, y pre This indicates the prediction result for driving style.

8. The end-to-end driving style recognition method based on multi-source sensor data according to claim 7, characterized in that, Specifically, S601 includes: S601A: The real-time driving record is projected onto a high-dimensional space using the first projection unit to obtain first driving projection data; S601B: The first driving projection data is convolved and activated using the local convolution unit and the activation function unit to obtain the first output feature: X conv =SiLU(W conv *X p ) Among them, X conv Let W represent the first output feature, SiLU represent the SiLU activation function, and W represent the first output feature. conv X represents the convolution kernel. p This represents the first driver's projection data, and * indicates a convolution operation; S601C: The hidden state and output of the first output feature are updated step by step at each time step using the selective state space unit to obtain the second output feature: X ssm =S6(X conv ) Among them, X ssm S6 represents the second feature, and S6 represents the selective state-space unit function. S601D: The real-time driving record is projected onto a high-dimensional space using the third projection unit to obtain second driving projection data; S601E: The second activation function unit is used to activate the two driving projection data to obtain the third output feature: X act =SiLU(X p ) Among them, X act Indicates the third output feature; S601F: The second output feature and the third output feature are fused by the channel fusion unit to obtain the fused feature: X fuse =X act ×X ssm Among them, X fuse Indicates fusion characteristics; S601G: Using the second projection unit, a back-projection operation is performed on the fused features to obtain the short-term and long-term spatiotemporal features of the driving style.

9. An end-to-end driving style recognition system based on multi-source sensor data, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the end-to-end driving style recognition method based on multi-source sensor data as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the end-to-end driving style recognition method based on multi-source sensor data as described in any one of claims 1 to 8.

Images