Automatic driving takeover system based on driving style

Abstract

The invention discloses an automatic driving takeover system based on a driving style, which relates to the technical field of automatic driving, extracts historical driving data of a driver by using a Monte Carlo method, constructs a deep learning algorithm model by using a convolutional neural network, and performs model training through digital traffic information in a networked data set. According to the method, the driving habit data is obtained, the convolutional neural network is trained by the preferred trust domain algorithm to obtain the man-machine co-driving grade situation data, the convolutional neural network is iteratively trained for multiple times, and the obtained trained convolutional neural network model can accurately discriminate the driving style, so that the accuracy of overall recognition of the driving style is improved. According to the system, the driving styles can be determined according to the driving habits of different drivers in different driving environments, automatic driving after taking over is matched to the driving mode of the corresponding driving style level, and the traffic safety and the driving comfort are further improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an autonomous driving takeover system based on driving style. Background Art

[0002] With the rapid development of autonomous driving technology, drivers have increasingly higher requirements for autonomous driving to take over manual driving, gradually transitioning their focus from taking over safety to the comfort of the driving experience.

[0003] Currently, in the automatic driving takeover method, most of the research takes into account the traffic conditions, road conditions and the driver's own operating behavior, but there are few studies on automatic driving takeover based on different driving styles of drivers, resulting in low overall driver comfort and emotional satisfaction after automatic driving takes over manually driven vehicles. In addition, the previous automatic driving data-driven methods have the problem of inaccurate identification of driving style, so that the automatic driving system cannot fully understand the driver's behavioral intentions after taking over the manually driven vehicle. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides an automatic driving takeover system based on driving style, comprising a driving style feature extraction module, a deep training module and an automatic driving module electrically connected in sequence; the driving style feature extraction module is used to fuse the driver's data features, and obtain the driver's driving habits by training and integrating a convolutional neural network; the driving style feature extraction module includes a vehicle-mounted radar, and also includes:

[0005] Networking devices, which are used to connect vehicles to the Internet, other vehicles, and road infrastructure through wireless communication technology;

[0006] Engine comprehensive analyzer, used to detect, analyze and judge the dynamic and static performance and technical status of the engine;

[0007] Four-wheel alignment instrument, used to detect the alignment parameters of automobile wheels and compare them with the original design parameters; diagnose vehicle discomfort symptoms through alignment angle measurement, and determine the target object's driving style based on the target object's driving behavior data;

[0008] Basic driving database, which collects data through data provided by connected devices;

[0009] Real-time driving database, used to collect data from four-wheel alignment, vehicle radar and engine comprehensive analyzer;

[0010] The deep training module includes a driving data training device, a GPU device and a driving training database. The driving data training device and the GPU device collect data from the basic driving database and the real-time driving database for training.

[0011] The autonomous driving module collects driver habit data and obtains average data to match the autonomous driving mode, which includes competitive driving, cooperative driving and altruistic driving.

[0012] The technical solution further defined in the present invention is:

[0013] Furthermore, for the driving style feature data, principal component analysis is performed on the driving style features of n sample stages, and the corresponding sample matrix is:

[0014]

[0015] Among them, p is the number of driving style features; n is the number of sample stages, that is, the number of samples; let the correlation coefficient matrix be R, then:

[0016]

[0017] Among them, r ij With X i and X j The correlation coefficient is expressed as follows:

[0018]

[0019]

[0020]

[0021]

[0022] Among them, X i and X j represents any two of the n driving style features, r ij Then the arbitrary two driving style features X i and X j The correlation coefficient between is the mean of the i-th column in matrix X, is the mean value of the jth column in matrix X; var(X i ) is used to characterize X i The degree of dispersion, that is, variance; var(X j ) is used to characterize X j The degree of dispersion, that is, the variance; cov(X i , X j ) is used to characterize X i and X j The degree of correlation between them, i.e., covariance;

[0023] Then, the eigenvalues ​​λ1, λ2, ..., λ of R are calculated by Matlab.p , a total of p eigenvalues, and λ1>λ2>…>λ p > 0, then the contribution rate of each principal component, that is, each driving style feature, is calculated as follows:

[0024]

[0025] Among them, f i is the contribution rate of the ith principal component. The first n principal components are selected as the n main eigenvalues ​​that can determine the driving style, that is, the driving style characteristics with a cumulative contribution rate of more than 80% and an eigenvalue greater than 1.

[0026] In the aforementioned driving style-based automatic driving takeover system, n main eigenvalues, i.e., driving style characteristic data, are determined as the average longitudinal acceleration, the average following distance, and the maximum value of the product of the steering wheel angle and the vehicle speed in each stage; the driving style characteristic data are determined based on statistical analysis of the vehicle status information in the corresponding stage, and the vehicle status information includes the vehicle's steering wheel angle, vehicle speed, longitudinal acceleration, lateral acceleration, and following distance.

[0027] In the aforementioned driving style-based automatic driving takeover system, the networking device extracts the driver's driving road conditions and driving environment conditions during vehicle driving and uploads them to the basic driving database. The driving style feature extraction module extracts the driver's historical driving data and uploads it to the driving data training device, so that the relationship between the driver's driving and automatic driving is established.

[0028] The aforementioned automatic driving takeover system based on driving style establishes a Monte Carlo prediction process model in the driving style feature extraction module, and uses a computer to uniformly generate n random numbers x1, x2, ..., x in the interval [a, b]. n , calculate the integrand value f(x1), f(x2), ..., f(x n ), and calculate the average value of the integrand;

[0029] Select the gradient descent method, and select the gradient descent according to the number of samples brought into each solution of the loss function L:

[0030] L(θ n )=L(θ n-1 +Δθ)≈L(θ n-1 )+L′(θ n-1 )Δθ

[0031] In order to make θ n : =θ n-1 -αΔθ n , take Δθ n = -αL′(θ n-1 );

[0032] Among them, θ n represents the model parameters after the nth iteration; α represents the learning rate; L′(θ n-1 ) is the loss function with parameters θ n-1 The gradient at L(θ n ) is the functional expression of the loss function;

[0033] That is, the iterative formula for gradient descent is obtained:

[0034] θ n : =θ n-1 -αL′(θ n-1 )

[0035] Select the adaptive gradient method and obtain a method for adaptively adjusting the learning rate by recording the direction and distance of each iteration:

[0036]

[0037] θ n : =θ n-1 -αΔθ n

[0038] Among them, Δθ n Denotes the loss function L(θ n ) in the parameter θ n and the gradient at random samples; ∈ is the minimum value; α represents the learning rate; is the parameter g i The gradient of the sum from 1 to n-1 in time, g i This is to manually set the initial value.

[0039] In the aforementioned driving style-based automatic driving takeover system, when establishing a Monte Carlo prediction process model in the driving style feature extraction module, the activation function tanh model is established as follows:

[0040]

[0041] Here, x represents the element in the hyperbolic tangent function; the tanh function processes arrays element by element. The function accepts both real and complex inputs, and all angles are expressed in radians.

[0042] The activation function Softmax model is established as:

[0043]

[0044] Among them, x iFor any real vector of length K, the Softmax function compresses it into a real vector of length K, with values ​​in the range of [0,1], and the sum of the elements in the vector is 1.

[0045] In the aforementioned driving style-based automatic driving takeover system, in the convolutional neural network, the input driving style feature data sequentially passes through the input layer, the first convolutional layer, the first average pooling layer, the second convolutional layer, the second average pooling layer, the third convolutional layer, the first fully connected layer, and the second fully connected layer;

[0046] The input layer is used to receive the input driving style feature data and extract the driver's driving road conditions and driving environment conditions; the first convolution layer uses multiple 5×5 filters with a step size of 1, no padding operation, outputs feature maps, and uses tanh as the activation function; the first average pooling layer uses a 2×2 pooling window with a step size of 2 to downsample, and the size of the feature map is halved; the second convolution layer uses multiple 5×5 filters with a step size of 1 to output feature maps, and uses tanh as the activation function; the second average pooling layer uses a 2×2 pooling window with a step size of 2 to further downsample, and the size of the feature map is reduced. The third convolutional layer uses multiple 5×5 filters with a step size of 1, no pooling, outputs a multi-dimensional feature vector, and uses tanh as the activation function; the feature vector in the first fully connected layer is flattened and connected to a fully connected layer with multiple neurons, and the activation function is tanh; the last fully connected layer in the second fully connected layer maps the multi-dimensional feature vector to multiple output neurons, each neuron corresponds to a classification label, and the activation function is Softmax, which is used to output the probability distribution of each category in multi-classification tasks; finally, the classification result is output through the Softmax layer to obtain the driver's driving habit characteristics.

[0047] In the aforementioned driving style-based automatic driving takeover system, in the convolutional neural network, a trust region algorithm is selected to train the convolutional neural network to obtain human-machine co-driving level situation data, and the convolutional neural network is iteratively trained multiple times to obtain a trained convolutional neural network model; the trust region algorithm model is established and expressed as:

[0048]

[0049] The above formula transforms minf(x, y) into a one-dimensional step-size problem, where By continuously finding the step size, we can further iterate to get new x and y; function m k (p) is the function f(x, y) at the point f(x k ,y k)After Taylor expansion, an approximate function is obtained, which is the first three terms after its expansion; Δk>0 represents the trust region radius; sk is the solution of the above equation, which becomes the trial step; It is first order. It is the second order, and the trained driving style data is obtained.

[0050] In the aforementioned driving style-based automatic driving takeover system, the trained convolutional neural network model includes a ReLU function, and its calculation formula is as follows:

[0051] ReLU: f(x) = max(0, x)

[0052] Among them, if the input data of x is greater than 0, then the input data is output as the output data, otherwise 0 is output.

[0053] In the aforementioned driving style-based automatic driving takeover system, the automatic driving module divides the human-machine co-driving level situation into levels, and determines the driver's driving style level according to the human-machine co-driving level situation, and the driver's driving style level includes low-level state, medium-level state and high-level state; when the driver's driving style level is in a low-level state, the trust region method is used to update the strategy, and the automatic driving mode is determined to be altruistic driving; when the driver's driving style level is in a medium-level state, the adaptive gradient method is used to determine the strategy that maximizes the activation function tanh and the ReLU function, and the automatic driving mode is determined to be cooperative driving; when the driver's driving style level is in a low-level state, the full gradient descent method is used to determine the solution, and the automatic driving mode is determined to be competitive driving.

[0054] The beneficial effects of the present invention are:

[0055] (1) In the present invention, the Monte Carlo method is used to extract the driver's historical driving data, and a convolutional neural network is used to construct a deep learning algorithm model. The model is trained through digital traffic information in the networked data set to obtain driving habit data. The trust region algorithm is preferably used to train the convolutional neural network to obtain human-machine co-driving level situation data. The convolutional neural network is then iteratively trained for multiple times to obtain a trained convolutional neural network model that can accurately distinguish driving style, thereby improving the accuracy of overall driving style recognition;

[0056] (2) In the present invention, different drivers are in different driving environments and can determine their driving style according to their own driving habits, and match the automatic driving after taking over to a driving mode corresponding to the driving style level, thereby further improving traffic safety and driving comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is the overall system block diagram of the present invention;

[0058] Figure 2 It is a schematic diagram of the overall process of the automatic driving takeover system in an embodiment of the present invention;

[0059] Figure 3 Schematic diagram of the structure of a convolutional neural network in an embodiment of the present invention;

[0060] Figure 4 The present invention is a flowchart of selecting a corresponding automatic driving mode according to different levels of human-machine co-driving situations. DETAILED DESCRIPTION

[0061] This embodiment provides an automatic driving takeover system based on driving style. The system determines the driving style of the driver according to the driver's own driving habits in different external driving environments, and matches the automatic driving after takeover to a driving mode corresponding to the driving style level, so as to further improve traffic safety and driving comfort.

[0062] like Figure 1 As shown, it includes a driving style feature extraction module, a deep training module and an automatic driving module which are electrically connected in sequence; the driving style feature extraction module includes a networking device, a basic driving database, an engine comprehensive analyzer, an on-board radar, a four-wheel alignment device and a real-time driving database; the deep training module includes a driving data training device (Field Programmable Gate Array, FPGA), a high-performance GPU device and a driving training database.

[0063] In the driving style feature extraction module, the networking device connects the vehicle to the Internet, other vehicles and road infrastructure through wireless communication technology, providing traditional functions such as navigation and entertainment, and introducing functions such as real-time traffic information, remote control, and intelligent driving assistance. The core advantage of the vehicle networking system lies in its powerful information processing and interaction capabilities, which can provide users with more accurate and personalized services and can be uploaded to the basic driving database.

[0064] The engine comprehensive analyzer is used to detect, analyze and judge the dynamic and static performance and technical status of the engine. It is mainly based on the basic principles of physical perspective and computer information processing technology. When performing four-wheel alignment, four target reflectors are usually installed on the four rims of the vehicle. The geometric figures on these reflectors are continuously photographed by a camera. The computer will analyze and calculate the changes in these figures to obtain the corresponding positioning parameters of the wheels and chassis. These parameters include the camber angle, caster angle, toe, etc. of the wheels, which together determine the driving stability of the vehicle and the wear of the tires.

[0065] In addition, the corresponding driving style is determined based on the driving behavior data of the target object. The process of identifying the driving style is based on the driving style characteristic data of the average longitudinal acceleration, the average following distance, and the maximum value of the product of the steering wheel angle and the vehicle speed. Therefore, it can accurately and in real time describe the driving style of the target object when turning, accelerating and decelerating; further, by combining the road condition category of the corresponding stage to identify the driving style, the accuracy of driving style recognition can be improved.

[0066] Vehicle-mounted radar is a radar system used in cars or other ground motor vehicles. It can detect obstacles, surrounding driving environment, etc., and detect the position and speed of surrounding objects by emitting electromagnetic waves and receiving their reflected waves. The radar system is mainly composed of three parts: transmitter, receiver and signal processor. The transmitter is responsible for generating and emitting electromagnetic waves of specific frequency, which will be reflected when encountering obstacles; the receiver is responsible for capturing these reflected electromagnetic waves; by analyzing the time delay and frequency change of the reflected wave, the signal processor can calculate the distance, direction and relative speed of the obstacle.

[0067] The four-wheel aligner is used to detect the alignment parameters of automobile wheels and compare them with the original design parameters. It can diagnose vehicle discomfort symptoms through alignment angle measurement, the average longitudinal acceleration, the average following distance and the maximum driving style characteristic data of the product of the steering wheel angle and the vehicle speed, such as deviation from the direction and trembling. It is mainly based on the basic principles of physical perspective and computer information processing technology. The four-wheel aligner installs target reflectors on the four rims of the vehicle, and when the wheels roll, the camera continuously shoots the geometric figures on these reflectors. The computer analyzes and calculates these images to obtain the corresponding alignment parameters of the wheels and chassis.

[0068] The basic driving database collects data through data provided by networked devices.

[0069] Real-time driving database, used to collect data from four-wheel alignment, vehicle radar and engine comprehensive analyzer.

[0070] In the deep training module, the driving data training device and the high-performance GPU device collect data from the basic driving database and the real-time driving database for training.

[0071] The working principle of high-performance GPU devices can be divided into the following steps: 1. Vertex processing: processing vertex coordinates, color, texture and other information in graphics data to generate vertex data; 2. Primitive processing: connecting vertex data to form primitives, such as triangles, line segments, etc.; 3. Rasterization: converting primitive data into pixel data; 4. Pixel processing: processing each pixel and calculating its color value; 5. Output: outputting the final image data.

[0072] Its application in driving data training devices is mainly based on its high programmability, parallel processing capabilities and low power consumption characteristics; it can be configured according to specific application requirements to achieve customized hardware logic, which makes it very suitable for processing large amounts of data and complex algorithms in autonomous driving systems.

[0073] The driving data training device can be used to accelerate key tasks such as image processing, sensor data preprocessing, and control system logic. It can speed up the execution of image classification and processing algorithms and improve the ability to recognize driving conditions through the characteristics of parallel processing and high-speed memory. In addition, it can also achieve hardware acceleration and reconstruction, provide high-performance real-time control, reduce system latency, and update functions and fix errors without changing the chip through reconfigurability.

[0074] The driving style characteristic data include the average longitudinal acceleration, the average following distance and the maximum value of the product of the steering wheel angle and the vehicle speed in each stage; specifically, the driving style characteristic data is determined based on statistical analysis of the vehicle status information in the corresponding stage, and the vehicle status information may specifically include the vehicle's steering wheel angle, vehicle speed, longitudinal acceleration, lateral acceleration and following distance and other information.

[0075] Taking the principal component analysis of driving style characteristics of n sample stages as an example, the corresponding sample matrix is:

[0076]

[0077] Among them, p is the number of driving style features; n is the number of sample stages, that is, the number of samples; let the correlation coefficient matrix be R, then:

[0078]

[0079] Among them, r ij With X i and X j The correlation coefficient is expressed as follows:

[0080]

[0081]

[0082]

[0083]

[0084] Among them, X i and X j represents any two of the n driving style features, r ij Then the arbitrary two driving style features X iand X j The correlation coefficient between is the mean of the i-th column in matrix X, is the mean value of the jth column in matrix X; var(X i ) is used to characterize X i The degree of dispersion, that is, the variance; similarly, var(X j ) is used to characterize X j The degree of dispersion, that is, the variance; cov(X i , X j ) is used to characterize X i and X j The degree of correlation between them is called covariance.

[0085] Then, the eigenvalues ​​λ1, λ2, ..., λ of R are calculated by Matlab. p , a total of p eigenvalues, and λ1>λ2>…>λ p > 0, the contribution rate of each principal component (i.e. each driving style feature) is calculated as follows:

[0086]

[0087] Among them, f i is the contribution rate of the ith principal component, and the first n principal components (i.e., driving style features with cumulative contribution rates of more than 80% and eigenvalues ​​greater than 1) are selected as n principal eigenvalues ​​that can determine the driving style; in this embodiment, based on the driving style features of the above-mentioned multiple sample stages, the n principal eigenvalues ​​(i.e., driving style feature sample data) finally determined are the maximum value of the longitudinal acceleration average value, the following distance average value, and the product of the steering wheel angle and the vehicle speed.

[0088] During vehicle driving, four-wheel alignment, vehicle radar and engine comprehensive analyzer can be used; when identifying driving style, based on the driving style feature data of the average longitudinal acceleration, the average following distance and the maximum value of the product of the steering wheel angle and the vehicle speed, it is possible to accurately and in real time describe the target object's driving style when turning, accelerating and decelerating; by identifying the driving style in combination with the road condition category of the corresponding stage, the accuracy of driving style recognition can be improved; the driving style feature extraction module extracts the driver's historical driving data and uploads it to the driving data training device.

[0089] The driving style feature extraction module is used to fuse the driver data features and the convolutional neural network (CNN) training is used to integrate the driver's driving habits, such as Figure 2 As shown, the specific steps include:

[0090] S1. In order to establish a relationship between the driver's driving and the automatic driving, the driver's driving data is extracted during the vehicle's driving process, and the driver's driving road conditions and driving environment conditions are extracted using a networked device and uploaded to the basic driving database.

[0091] S2. Establish a Monte Carlo prediction process model in the driving style feature extraction module (based on the autonomous driving decision-making problem); the driving style feature extraction module fuses the driver's data features and uses convolutional neural network (CNN) training and integration to obtain the driver's driving style; using the Monte Carlo method, the use of random numbers (or pseudo-random numbers) can more realistically describe the characteristics of things with random properties and the physical experimental process. It is less restricted by geometric conditions, the convergence speed is independent of the dimension of the problem, and it has the ability to calculate multiple solutions and multiple unknown quantities at the same time. The error is easy to determine, the program structure is simple, and it is easy to predict the model.

[0092] Using the untitled command of Matlab software, n random numbers x1, x2, ..., x are uniformly generated on the interval [a, b] using a computer. n , calculate the integrand value f(x1), f(x2), ..., f(x n ) and calculate the average of the integrand values.

[0093] Select the gradient descent method. According to the number of samples brought into the loss function L each time, select gradient descent (calculate the loss of all samples):

[0094] L(θ n )=L(θ n-1 +Δθ)≈L(θ n-1 )+L′(θ n-1 )Δθ

[0095] In order to make θ n : =θ n-1 -αΔθ n , take Δθ n = -αL′(θ n-1 );

[0096] Among them, θ n represents the model parameters after the nth iteration; α represents the learning rate. It is usually recommended to start with a smaller learning rate and gradually adjust it according to the actual effect; L′(θ n-1 ) is the loss function with parameters θ n-1 The gradient at L(θ n ) is the functional expression of the loss function;

[0097] That is, the iterative formula for gradient descent is obtained:

[0098] θ n : =θn-1 -αL′(θ n-1 ).

[0099] Select the adaptive gradient method, which records the direction and distance of advance in each iteration, so that there is a method to adaptively adjust the learning rate for different problems:

[0100]

[0101] θ n : =θ n-1 -αΔθ n

[0102] Among them, Δθ n Denotes the loss function L(θ n ) in the parameter θ n and the gradient at the random sample; ∈ is the minimum value, which is a small constant used to prevent the denominator from being zero. When calculating the learning rate, it will be added to the cumulative square gradient. ∈ is generally 1e-7; α represents the learning rate. It is usually recommended to start with a smaller learning rate; is the parameter g i The gradient of the sum from 1 to n-1 in time, g i This is to manually set the initial value.

[0103] As the number of iterations increases, the learning rate gradually decreases, which is "intuitively" correct: the closer you get to the optimal solution, the flatter the "slope" of the function becomes, and you must go slower to ensure that you do not cross the optimal solution.

[0104] S3. In the driving style feature extraction module, a convolutional neural network is built using a deep learning algorithm, the convolutional neural network is optimized for the human-vehicle co-driving level situation, and the convolutional neural network is iteratively trained multiple times to obtain a trained convolutional neural network.

[0105] like Figure 3 As shown, in the convolutional neural network, the input driving style feature data passes through the input layer, the first convolution layer, the first average pooling layer, the second convolution layer, the second average pooling layer, the third convolution layer, the first fully connected layer and the second fully connected layer in sequence.

[0106] The workflow of a convolutional neural network consists of the following steps:

[0107] S3.1. The function of the input layer is to receive the input driving style feature features and driving behavior data. The input size here means that the network processes a single-channel (such as grayscale) image with the same pixel width and height, collects information about the surrounding roads obtained by the on-board camera and radar, and extracts the driver's driving conditions and driving environment.

[0108] In the driving style feature extraction module, the basic driving database is a system that collects and stores driving-related data, which usually includes driver information, vehicle information, driving behavior data, and related road and traffic environment information.

[0109] A Monte Carlo prediction process model is established in the driving style feature extraction module, and the collected information may include: driver information, vehicle information, driving behavior data, road and traffic environment information.

[0110] The driver information includes the driver's personal information, such as name, age, gender, driver's license type and validity period, as well as the driver's historical driving records, such as violation records, accident records, etc.

[0111] Vehicle information refers to the basic information of the vehicle, such as vehicle model, license plate number, vehicle identification number (VIN), annual vehicle inspection status, and vehicle maintenance and repair records.

[0112] Driving behavior data refers to the records of the driver's driving habits, such as average speed, number of emergency brakes, number of emergency accelerations, speeding records, etc. These data help to evaluate the driver's driving skills and driving style.

[0113] The road and traffic environment information refers to the road conditions, weather conditions, traffic flow, etc. encountered during driving. This information helps to analyze the impact of the driving environment on driving behavior.

[0114] By collecting and analyzing this data, the Basic Driving Database can provide valuable information to traffic management departments, insurance companies, automakers and drivers.

[0115] Real-time driving database A real-time driving database is a system that collects, processes and analyzes real-time driving data. The database usually contains the vehicle's position, speed, direction, acceleration, throttle and brake usage, and driver behavior data such as attention level, reaction time, etc.

[0116] The collection may include: road traffic conditions, autonomous driving data, fleet management, and vehicle performance monitoring.

[0117] The road traffic conditions refer to the real-time driving database that provides key information for the intelligent transportation system, such as traffic flow, congestion and accident warnings.

[0118] The autonomous driving data refers to the data collected in the real-time driving database. The real-time driving database is an important data source for autonomous driving vehicle learning and decision-making. By collecting and analyzing driving data, the autonomous driving system can continuously learn and optimize driving strategies to improve the safety and efficiency of autonomous driving.

[0119] Fleet management refers to the real-time driving database that can help fleet managers monitor vehicle locations, optimize route planning, monitor driver behavior, improve fleet operating efficiency, and reduce operating costs.

[0120] Vehicle performance monitoring refers to the real-time driving database that can monitor the vehicle's operating status, including the engine, braking system, tire wear, etc., to help car owners and maintenance personnel detect and solve vehicle problems in a timely manner.

[0121] S3.2. The first convolutional layer uses multiple 5×5 filters (or convolution kernels) with a step size of 1, no padding operation, and the size of the output feature map; tanh (hyperbolic tangent) is used as the activation function to extract the underlying features of the image and extract features from the input data; the convolutional layer performs convolution operations on the data using a set of learnable convolution kernels (or filters). These convolution kernels slide on the input data and calculate the weighted sum of the local area to generate a feature map.

[0122] S3.3. The first average pooling layer uses a 2×2 pooling window with a step size of 2 for downsampling, and the size of the feature map is halved. To prevent overfitting, the data dimension is reduced by calculating the average value of each small area in the input feature map. This process can help the network extract more robust features and reduce the number of model parameters and the amount of computation, which helps prevent overfitting.

[0123] S3.4, the second convolutional layer uses multiple 5×5 filters with a step size of 1, the size of the output feature map, and the activation function is still tanh, which is used to extract the underlying features of the image and collect image information through deep training.

[0124] S3.5, the second average pooling layer is a 2×2 pooling window with a step size of 2, which further downsamples the size of the output feature map to reduce the data dimension in order to prevent overfitting of image information collected by deep training.

[0125] S3.6, the third convolutional layer has multiple 5×5 filters with a step size of 1. No pooling is performed, and a multi-dimensional feature vector is directly output. Tanh is used as the activation function.

[0126] S3.7. The feature vector of the first fully connected layer is flattened and connected to a fully connected layer with 84 neurons, which summarizes the underlying features and information of the image obtained by the convolutional layer and the pooling layer. It is transformed nonlinearly through the activation function tanh, which can combine the feature information from the previous layers to form a more abstract and advanced representation.

[0127] S3.8. The last fully connected layer in the second fully connected layer maps the multi-dimensional feature vector to multiple output neurons, each neuron corresponds to a classification label; the activation function is Softmax, which is used to output the probability distribution of each category in multi-classification tasks and map the integrated features to the final output category.

[0128] S3.9. The convolutional layer is used to extract spatial features, the pooling layer is used to reduce the number of parameters and the amount of calculation, the fully connected layer is used to perform high-level abstraction of features, and finally the Softmax layer is used to output the classification results to obtain the driver's driving habit characteristics and identify the driving style.

[0129] Fusion of driving data features through convolutional layers; methods for evaluating and understanding the driver's driving behavior in real time or near real time using data collection. This method usually involves collecting various data during the driving process, including but not limited to: vehicle dynamic data (such as vehicle speed, acceleration, braking force, steering angle, etc.), which can reflect the driver's driving habits and skills; driving environment data (including road conditions, traffic flow, weather conditions, etc.), which help analyze the impact of the driving environment on driving style; driver behavior data (such as the driver's attention level, reaction time, driving decision-making, etc.), which can reflect the driver's mental state and driving attitude.

[0130] By dynamically analyzing the driving style, the following steps are involved in obtaining the driving style:

[0131] Driving behavior assessment: Real-time monitoring of the driver's driving behavior, such as sudden acceleration, sudden braking, speeding, etc., promptly detects unsafe driving behavior and provides feedback to help the driver improve his driving habits.

[0132] Personalized driving training: Provide personalized driving training suggestions based on the driver's behavior data, and use convolutional neural networks to train driving data, such as improving specific driving skills and enhancing the ability to respond to specific driving scenarios.

[0133] Risk assessment and management: Insurance companies and fleet managers can use driving style analysis to assess risks, formulate more reasonable insurance rates or management strategies, and reduce accident risks.

[0134] Vehicle performance optimization: Automakers can use driving style data to optimize vehicle design, improve vehicle handling and safety, and meet the needs of different driving styles.

[0135] Dynamic analysis of driving style usually relies on advanced sensor technology (such as GPS, accelerometers, cameras, etc.) and data processing technology (such as big data analysis, machine learning, etc.) to achieve efficient data collection and analysis.

[0136] Obtain human-machine co-driving level situation data, iterate the convolutional neural network for multiple times, and obtain the trained convolutional neural network model; in order to obtain driving style data, the optimization method adopts the trust region algorithm:

[0137]

[0138] The above formula transforms minf(x, y) into a one-dimensional step-size problem, where By continuously finding the step size, we can further iterate to get new x and y; function m k (p) is the function f(x, y) at the point f(x k ,y k )After Taylor expansion, an approximate function is obtained, which is the first three terms after its expansion; Δk>0 represents the trust region radius; sk is the solution of the above equation, which becomes the trial step; It is first order. It is second order.

[0139] S4. Generate autonomous driving decisions based on the trained convolutional neural network model (the driver authorizes autonomous driving and the driving mode is matched based on the generated autonomous driving decision).

[0140] The deep learning algorithm builds a convolutional neural network; the convolutional neural network is used for training, in which the ReLU function is a common activation function. Compared with the traditional activation function, the ReLU function can still maintain good results in the case of a large amount of data, and its operation speed is also faster than the traditional function. It can achieve the purpose of parameter sparsity by simply thresholding. The calculation formula of the ReLU function is as follows:

[0141] ReLU: f(x) = max(0, x)

[0142] This formula means that if the input data x is greater than 0, then the input data is output as the output data, otherwise it is output 0. By using the ReLU function, it can be ensured that the output data will not approach saturation due to the increase in input data.

[0143] In the convolutional neural network layer, the neural network obtains and extracts features through convolution kernels; multiple convolution kernels can be used for feature extraction in each layer, and each convolution kernel extracts a feature. The higher the level of the convolution layer, the closer the extracted features are to the features of the object itself; the convolution kernel is an n×n×r weight matrix, where n is the size of the convolution kernel and r is the depth of the convolution kernel. The training integration obtains the driver's driving habits.

[0144] The depth of the convolution kernel should be consistent with the depth of the input data. In the first convolution layer, the depth of the convolution kernel should be consistent with the depth of the input image.

[0145] The driving habit data module collects driver habit data and obtains average data to facilitate the matching of autonomous driving modes. The autonomous driving modes include: competitive driving, cooperative driving, and altruistic driving.

[0146] The activation function tanh model is established as:

[0147]

[0148] Here, x represents the element in the hyperbolic tangent function; the tanh function processes arrays element by element. The function accepts both real and complex inputs, and all angles are expressed in radians.

[0149] The activation function Softmax model is established as:

[0150]

[0151] Among them, x i For any real vector of length K, the Softmax function compresses it into a real vector of length K, with values ​​in the range of [0, 1], and the sum of the elements in the vector is 1.

[0152] The max function only outputs the maximum value, but the Softmax function ensures that smaller values ​​have smaller probabilities and are not discarded directly. The denominator of the Softmax function combines all the factors of the original output value, which means that the various probabilities obtained by the Softmax function are related to each other.

[0153] According to the trained convolutional neural network model, the human-machine co-driving level situation data is obtained, and the autonomous driving module classifies the human-machine co-driving level situation, such as Figure 4 The specific classification is as follows:

[0154] S4.1. According to the human-machine co-driving level situation, the driver's driving style level is determined to be in a low level state, and the strategy is updated using the trust domain method.

[0155] Altruistic driving means that while considering one's own interests, one also considers the interests of other road users as much as possible. Altruistic driving enables the driver to not only optimize its own utility when making decisions, but also consider the impact on other vehicles and pedestrians. For example, an autonomous vehicle may slow down when merging or changing lanes to avoid interference or danger to other vehicles. Altruistic driving is a driving behavior that considers the interests of other road users, even if doing so may sacrifice some of its own interests. It is used to train autonomous vehicles to take behaviors that are conducive to overall traffic flow and safety when faced with complex and competitive driving scenarios, such as slowing down to allow other vehicles to merge or pass through intersections.

[0156] S4.2. According to the human-machine co-driving level situation, the driver's driving style level is determined to be in the medium level state, and the adaptive gradient method is used to determine the strategy to maximize the activation function tanh and ReLU function.

[0157] Because the activation function tanh and ReLU function and other layers in the deep learning model are linear function fitting, even if a very deep network is used for fitting, it is still unavoidable to have linear characteristics and cannot perform nonlinear modeling. By adding a nonlinear activation function unit, when the output layer of the linear function passes through the nonlinear activation unit, its output presents a nonlinear change. In this way, after multiple layers of fitting, the nonlinear modeling operation of the input can be completed; at the same time, it can also play the role of a feature combination.

[0158] Cooperative driving refers to some form of communication and coordination between vehicles, working together to improve the efficiency and safety of the entire transportation system. Cooperative driving can be achieved through vehicle-to-vehicle communication, which enables vehicles to share information such as speed, location and intention, so as to jointly plan driving routes and speeds and reduce conflicts and congestion. It emphasizes the interaction and coordination between vehicles to improve the efficiency and safety of the entire transportation system. In cooperative driving, vehicles will consider the behavior and intentions of other vehicles and coordinate actions by exchanging information (such as through vehicle-to-vehicle communication), such as changing lanes together or maintaining a stable convoy on the highway.

[0159] S4.3. According to the co-driving situation of human and machine, the driving style level of the driver is determined to be at a high level, and the solution is determined by the full gradient descent method, and the automatic driving mode is determined to be competitive driving. The full gradient descent algorithm calculates all sample errors in the training set, sums them up and takes the average value as the objective function.

[0160] Competitive driving usually refers to the independent decision-making and actions made by each vehicle in road traffic for its own interests (such as reaching the destination quickly, saving time, etc.). This driving style may lead to reduced traffic flow efficiency; competitive driving can be reflected in the autonomous driving system giving priority to the optimal path and speed of its own vehicle, without considering the behavior and needs of other vehicles; it means that the behavior of the vehicle on the road is mainly to maximize its own interests, rather than considering the overall traffic flow or the interests of other vehicles; in this driving mode, the vehicle may take more aggressive or self-centered actions, such as forced lane changes, accelerated overtaking, etc., to reduce travel time or improve fuel efficiency.

[0161] The system of this embodiment uses the Monte Carlo method to extract the driver's historical driving data, uses a convolutional neural network to build a deep learning algorithm model, and trains the model through digital traffic information in the networked data set to obtain driving habit data. The convolutional neural network is trained using the trust region algorithm to obtain human-machine co-driving level situation data, and then the convolutional neural network is iteratively trained multiple times to obtain a trained convolutional neural network model that can accurately distinguish driving style, thereby improving the accuracy of overall driving style recognition.

[0162] The system of this embodiment can determine the driving style of different drivers in different driving environments according to their own driving habits, and match the automatic driving after taking over to the driving mode of the corresponding driving style level, so as to further improve traffic safety and driving comfort.

[0163] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solution formed by equivalent replacement or equivalent transformation falls within the protection scope required by the present invention.

Claims

1. An automatic driving takeover system based on driving style, characterized in that: The system comprises a driving style feature extraction module, a deep training module and an automatic driving module which are electrically connected in sequence; the driving style feature extraction module is used to fuse the driver's data features and obtain the driver's driving habits by training and integrating a convolutional neural network; the driving style feature extraction module comprises an on-board radar and also comprises: Networking devices, which are used to connect vehicles to the Internet, other vehicles, and road infrastructure through wireless communication technology; Engine comprehensive analyzer, used to detect, analyze and judge the dynamic and static performance and technical status of the engine; Four-wheel alignment instrument, used to detect the alignment parameters of automobile wheels and compare them with the original design parameters; diagnose vehicle discomfort symptoms through alignment angle measurement, and determine the target object's driving style based on the target object's driving behavior data; Basic driving database, which collects data through data provided by connected devices; Real-time driving database, used to collect data from four-wheel alignment, vehicle radar and engine comprehensive analyzer; The deep training module includes a driving data training device, a GPU device and a driving training database. The driving data training device and the GPU device collect data from the basic driving database and the real-time driving database for training. The autonomous driving module collects driver habit data and obtains average data to match the autonomous driving mode, which includes competitive driving, cooperative driving and altruistic driving.

2. The automatic driving takeover system based on driving style according to claim 1, characterized in that: For the driving style feature data, principal component analysis is performed on the driving style features of n sample stages, and the corresponding sample matrix is: Among them, p is the number of driving style features; n is the number of sample stages, that is, the number of samples; let the correlation coefficient matrix be R, then: Among them, r ij With X i and X j The correlation coefficient is expressed as follows: Among them, X i and X j represents any two of the n driving style features, r ij Then the arbitrary two driving style features X i and X j The correlation coefficient between is the mean of the i-th column in matrix X, is the mean value of the jth column in matrix X; var(X i ) is used to characterize X i The degree of dispersion, that is, variance; var(X j ) is used to characterize X j The degree of dispersion, that is, the variance; cov(X i , X j ) is used to characterize X i and X j The degree of correlation between them, i.e., covariance; Then, the eigenvalues ​​λ1, λ2, ..., λ of R are calculated by Matlab. p , a total of p eigenvalues, and λ1>λ2>…>λ p > 0, then the contribution rate of each principal component, that is, each driving style feature, is calculated as follows: Among them, f i is the contribution rate of the ith principal component. The first n principal components are selected as the n main eigenvalues ​​that can determine the driving style, that is, the driving style characteristics with a cumulative contribution rate of more than 80% and an eigenvalue greater than 1.

3. The automatic driving takeover system based on driving style according to claim 2, characterized in that: The n main eigenvalues, i.e., driving style characteristic data, are determined as the average longitudinal acceleration, the average following distance, and the maximum value of the product of the steering wheel angle and the vehicle speed in each stage; the driving style characteristic data are determined based on statistical analysis of the vehicle status information in the corresponding stage, and the vehicle status information includes the vehicle's steering wheel angle, vehicle speed, longitudinal acceleration, lateral acceleration, and following distance.

4. The automatic driving takeover system based on driving style according to claim 1, characterized in that: The networking device extracts the driver's driving road conditions and driving environment during vehicle driving and uploads them to the basic driving database. The driving style feature extraction module extracts the driver's historical driving data and uploads it to the driving data training device, so that the relationship between the driver's driving and automatic driving is established.

5. The automatic driving takeover system based on driving style according to claim 1, characterized in that: A Monte Carlo prediction process model is established in the driving style feature extraction module, and n random numbers x1, x2, ..., x are uniformly generated by a computer in the interval [a, b]. n , calculate the integrand value f(x1), f(x2), ..., f(x n ), and calculate the average value of the integrand; Select the gradient descent method, and select the gradient descent according to the number of samples brought into each solution of the loss function L: L(θ n )=L(θ n-1 +Δθ)≈L(θ n-1 )+L′(θ n-1 )Dth To make θ n : = θ n-1 -αΔθ n , take Δθ n = -αL′(θ n-1 ); Among them, θ n represents the model parameters after the nth iteration; α represents the learning rate; L′(θ n-1 ) is the loss function with parameters θ n-1 The gradient at L(θ n ) is the functional expression of the loss function; That is, the iterative formula for gradient descent is obtained: i n :=θ n-1 -αL′(θ n-1 ) Select the adaptive gradient method and obtain a method for adaptively adjusting the learning rate by recording the direction and distance of each iteration: i n :=θ n-1 -αΔθ n Among them, Δθ n Denotes the loss function L(θ n ) in the parameter θ n and the gradient at random samples; ∈ is the minimum value; α represents the learning rate; is the parameter g i The gradient of the sum from 1 to n-1 in time, g i This is to manually set the initial value.

6. The automatic driving takeover system based on driving style according to claim 5, characterized in that: When establishing the Monte Carlo prediction process model in the driving style feature extraction module, the activation function tanh model is established as follows: Here, x represents the element in the hyperbolic tangent function; the tanh function processes arrays element by element. The function accepts both real and complex inputs, and all angles are expressed in radians. The activation function Softmax model is established as: Among them, x i For any real vector of length K, the Softmax function compresses it into a real vector of length K, with values ​​in the range of [0, 1], and the sum of the elements in the vector is 1.

7. A method according to claim 1, characterized in that: In the convolutional neural network, the input driving style feature data sequentially passes through the input layer, the first convolutional layer, the first average pooling layer, the second convolutional layer, the second average pooling layer, the third convolutional layer, the first fully connected layer and the second fully connected layer; The input layer is used to receive the input driving style feature data and extract the driver's driving road conditions and driving environment conditions; the first convolution layer uses multiple 5×5 filters with a step size of 1, no padding operation, outputs feature maps, and uses tanh as the activation function; the first average pooling layer uses a 2×2 pooling window with a step size of 2 to downsample, and the size of the feature map is halved; the second convolution layer uses multiple 5×5 filters with a step size of 1 to output feature maps, and uses tanh as the activation function; the second average pooling layer uses a 2×2 pooling window with a step size of 2 to further downsample, and the size of the feature map is reduced. The third convolutional layer uses multiple 5×5 filters with a step size of 1, no pooling, outputs a multi-dimensional feature vector, and uses tanh as the activation function; the feature vector in the first fully connected layer is flattened and connected to a fully connected layer with multiple neurons, and the activation function is tanh; the last fully connected layer in the second fully connected layer maps the multi-dimensional feature vector to multiple output neurons, each neuron corresponds to a classification label, and the activation function is Softmax, which is used to output the probability distribution of each category in multi-classification tasks; finally, the classification result is output through the Softmax layer to obtain the driver's driving habit characteristics.

8. A method according to claim 7, characterized in that: In the convolutional neural network, a trust region algorithm is selected to train the convolutional neural network to obtain human-machine co-driving level situation data, and the convolutional neural network is iteratively trained multiple times to obtain a trained convolutional neural network model; the trust region algorithm model is established as: The above formula transforms minf(x, y) into a one-dimensional step-size problem, where By continuously finding the step size, we can further iterate to get new x and y; function m k (p) is the function f(x, y) at the point f(x k ,y k )After Taylor expansion, an approximate function is obtained, which is the first three terms after its expansion; Δk>0 represents the trust region radius; sk is the solution of the above equation, which becomes the trial step; It is first order. It is the second order, and the trained driving style data is obtained.

9. A method according to claim 8, characterized in that: The trained convolutional neural network model includes a ReLU function, and its calculation formula is as follows: ReLU: f(x) = max(0, x) Among them, if the input data of x is greater than 0, then the input data is output as the output data, otherwise 0 is output.

10. A method according to claim 1, characterized in that: The automatic driving module classifies the human-machine co-driving level situation, and determines the driver's driving style level according to the human-machine co-driving level situation, and the driver's driving style level includes a low-level state, a medium-level state and a high-level state; when the driver's driving style level is a low-level state, the trust domain method is used to update the strategy, and the automatic driving mode is determined to be altruistic driving; when the driver's driving style level is a medium-level state, the adaptive gradient method is used to determine the strategy that maximizes the activation function tanh and the ReLU function, and the automatic driving mode is determined to be cooperative driving; when the driver's driving style level is a low-level state, the full gradient descent method is used to determine the solution, and the automatic driving mode is determined to be competitive driving.

Images