Privacy protection-oriented cloud side-end collaborative driving style identification method

Abstract

The invention discloses a privacy protection-oriented cloud side-end collaborative driving style identification method, which comprises the following steps of: screening out effective driving data from a side-end collection machine through data preprocessing, processing the effective driving data into windows with the same size according to requirements, putting the batch of labeled data on a side-end model to train the side-end model and obtain probability distribution, and carrying out cloud side-end collaborative driving style identification. And constructing a probability distribution integration framework, and calculating a global probability distribution according to the probability distribution given by each side end. And guiding cloud model training according to the obtained global probability distribution because the cloud model cannot have data with labels. After cloud model training is completed, probability distribution is obtained according to a batch of driving data without labels, training of each side end model is guided according to the probability distribution, and finally the trained model is put into a driving style recognition task of new data to complete driving style recognition. And accurate identification of the driving style is realized through more comprehensive global probability distribution and an excellent cloud edge-end cooperation model.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving and data mining technology, and in particular to a cloud-edge-end collaborative driving style recognition method for privacy protection. Background Art

[0002] In recent years, the driverless car industry has received high attention and key support from the industry, which has promoted the development of the driverless car industry.

[0003] In the future, the current situation of human-machine collaborative driving vehicles will continue to exist. In order to achieve effective collaboration between people, vehicles and roads, it is necessary to carefully analyze human driving behavior and driving style so that the assisted driving system can better complete the assisted driving task. However, there are several difficulties in identifying human driving behavior and driving style: (1) The contradiction between the recognition task's demand for high-quality data and user privacy protection. In order to achieve high-precision recognition effects, a large amount of location-related driving data is often required, and this driving data contains the driver's privacy information, such as when they have been to a certain place. How to use this data to train the recognition model without infringing on user privacy is a major challenge. (2) The terminal device has weak computing power and cannot run highly complex recognition models. In addition, the real-time requirements for driving style recognition are high, and the inference speed of complex models is slow, which is not suitable for use in terminal devices.

[0004] Therefore, this application intends to propose a set of driving style recognition technology methods for privacy protection in response to the above two difficulties. First, a cloud-edge collaborative driving behavior model training method is proposed using the federated learning mechanism. At the edge, such as a car, a local recognition model is trained using privacy data. Then, the cloud server uses the federated learning mechanism to learn the parameters of each local model and infer and update the cloud model. The cloud model is then deployed to the edge to achieve the purpose of sharing the model without sharing user data. Privacy protection is achieved. Among them, a cloud model with strong learning ability is designed, and the probability distribution predicted by the learning edge model is distilled to the edge model, so that a simple model with superior performance can be achieved at the edge where a complex model cannot be arranged. In the process of integrating the edge into the cloud, a model quality score must be evaluated for each edge as the weight of the edge model. The simplest method is to calculate the amount of edge data and the strength of the model resolution ability. However, the data collected by the edge device will inevitably be unbalanced, which will affect the model resolution ability.

[0005] In order to solve this problem, the present invention proposes a scoring standard for the quality of edge models, performs fine-grained calculations on the accuracy of driving style recognition, and adds weight factors to the driving style recognition results to generate a global probability distribution when the edge model is integrated into the cloud model. The present invention can not only promote theoretical innovations in related technologies, but also effectively solve practical problems in the application of intelligent vehicle systems in driving behavior recognition, and has important economic and social value in promoting intelligent vehicle assisted driving systems. Summary of the invention

[0006] The purpose of the present invention is to provide a privacy-protected cloud-edge collaborative driving style recognition method, which aims to improve the driving style recognition capability of the edge model through a cloud-edge collaborative framework based on the driving data collected by the edge device, so as to achieve better driving style recognition effect with a simple architecture.

[0007] A privacy-preserving cloud-edge collaborative driving style recognition method includes the following steps:

[0008] 1) Preprocessing: Collect a certain amount of driving data on each edge device, process the long-term driving data, including extracting features and dividing windows, normalizing the driving data, and finally dividing the training set and test set to obtain driving data feature segments and driving styles; (driving data has multiple features, and after feature extraction, effective driving data features are obtained, and driving data feature segments are obtained after dividing the driving data features into windows);

[0009] 2) Build an edge architecture and train an edge model. Use a relatively simple neural network model on the edge. Based on the driving data feature segments and driving style categories obtained in step 1), perform data enhancement on the data, add minority class samples (minority class samples refer to a collection of driving data feature segments with a small number of driving style categories), and train the edge model based on the processed driving data feature segments.

[0010] 3) Build a cloud model and propose an innovative model architecture that combines convolutional neural network (CNN), bidirectional long short-term memory network (BiLSTM) and autoencoder, called ACB (Autoencoder-enhanced Convolutional BiLSTM), for privacy-preserving driving style recognition.

[0011] 4) Build a cloud-edge collaborative framework to simulate some unlabeled driving data (after processing the driving data, driving data feature segments will be obtained. Each driving data feature segment corresponds to a driving style. This driving style is the label of this driving data segment. Unlabeled driving data refers to driving data feature segments without driving style, which need to be labeled by researchers). Use the edge model to predict the classification results, and combine the edge model quality score to integrate the category probability distribution predicted by the edge into the cloud. That is, use the global probability distribution to guide the cloud model training, and realize the distillation of knowledge obtained by the edge model to the cloud;

[0012] 5) After the cloud model training is completed, the cloud model is deployed to the edge. The driving style recognition results are obtained on the cloud model based on the unlabeled driving data obtained on the edge. The knowledge obtained by the cloud model is distilled to the edge model, and the complex cloud model is used to guide the training of the edge model.

[0013] Finally, the distilled edge model is used for new driving data to complete driving style recognition.

[0014] In the present invention, compared with the traditional driving style recognition model, this model builds a cloud-edge collaborative framework to protect the privacy of driving data, in which most of the data processing and feature extraction are completed on the edge, and only the processed feature data and prediction results need to be uploaded to the cloud. This greatly reduces the amount of data transmission, reduces the network burden, and reduces transmission costs and delays. In addition, the prediction results of the edge model in the cloud-edge collaborative framework can be used for training the cloud model to achieve knowledge distillation and integrated learning, and improve the accuracy and robustness of the cloud model; at the same time, the knowledge of the cloud model can also be reverse distilled to the edge model, so that it performs better in local recognition. This model also designs an integration module to generate a more comprehensive global probability distribution, which helps the cloud model to better learn the data distribution on the edge. Combining the above architecture and technology, the edge model performs better in driving style recognition tasks.

[0015] In step 1), the driving data features include vehicle ID, time, driving speed features, driving acceleration features and road condition features.

[0016] In step 3), the proposed ACB model includes the following main parts:

[0017] Convolutional Neural Network (CNN) feature extractor, Bidirectional Long Short-Term Memory (BiLSTM) time series processor, classifier head, autoencoder embedding learning module.

[0018] In step 4), the cloud-edge-device collaborative architecture specifically includes:

[0019] Edge-to-cloud distillation branch, cloud-to-edge distillation branch.

[0020] In step 4), in the edge-to-cloud distillation, firstly, the cloud-simulated data is used to make all edges obtain the corresponding probability distribution, which specifically includes:

[0021] The classification accuracy is calculated for the edge classification results. The accuracy is calculated for each driving style separately. The specific method is to count only when the predicted value is the same as the true value, and then divide it by the number of true values:

[0022]

[0023] Where Ac[i] is the accuracy of driving style i, N is the total number of samples, i is the driving style category, and y true,j is the true value of the jth sample, y pred,j is the predicted value of the jth sample, ∧ represents the AND operation, which is used to connect two values. The entire expression is true only when both values ​​are i. 1(·) is an indicator function, which takes the value 1 when the condition is true, otherwise it takes the value 0;

[0024] Calculate the weight w[i] of driving style category i:

[0025]

[0026] Where counts[i] represents the number of samples of driving style i, total represents the total number of samples, and num represents the number of driving style categories. When counts[i] is less than the threshold of the sample category, that is, the total number of samples divided by 2 times the number of categories, the weight of category i is adjusted to α, otherwise the weight is adjusted to 0.

[0027] Calculate the weighted accuracy wacc:

[0028]

[0029] Multiply the accuracy of each category by the corresponding category weight, add the products and divide by the weight sum to get the final weighted accuracy of each edge.

[0030] In step 4), the model quality score qs of each edge model is calculated:

[0031] qs=w d ·ds+w c ·wacc

[0032] Among them, ds is the data size, wacc is the weighted driving style fine-grained recognition accuracy, and w d is the data volume weight, w c is the classification accuracy weight.

[0033] Use each edge probability distribution p combined with the corresponding model quality score qs to calculate the final probability distribution cp:

[0034]

[0035] N represents the number of edges, p k and qs k Represents the probability distribution and model quality score corresponding to the kth edge.

[0036] The obtained global probability distribution is normalized to obtain cp′ so that the sum is one:

[0037]

[0038] In step 4), the probability distribution obtained by edge model integration guides the cloud model training, i.e., the distillation operation, which specifically includes:

[0039] According to the global probability distribution cp′ of the edge model and the driving style recognition result P of the cloud model cloud Use the KL divergence formula to calculate the distillation loss L distill :

[0040] Calculate the loss:

[0041]

[0042] Among them, cp' i and are the probabilities corresponding to the i-th class on the edge and in the cloud, respectively, scaled using the temperature parameter T;

[0043] The reconstruction loss is measured using mean squared error (MSE):

[0044]

[0045] in is the reconstructed vector and z is the original vector.

[0046] The overall minimization objective is defined as the combination of the reconstruction objective and the classification objective. For this autoencoder-based regularization with supervised learning, the unified loss function L u becomes:

[0047] L u =L distill +L r

[0048] Use gradient descent to update the parameters θ of the cloud model cloud :

[0049]

[0050] η is the learning rate, Represents the cloud model parameter θ cloud The gradient of the model is used to update the parameters of the model so that the loss function is gradually reduced, thereby optimizing the performance of the model.

[0051] In step 5), the knowledge acquired by the cloud model is distilled to the edge model, which includes:

[0052] According to the probability distribution P of the cloud model cloud And the edge model driving style recognition results Use the KL divergence formula to calculate the distillation loss of the i-th edge

[0053]

[0054] is the probability of the jth class in the cloud, is the probability of the jth category of the ith edge.

[0055] Use gradient descent to update the parameters of the i-th edge model:

[0056]

[0057] The present invention allows a complex cloud model to learn the distribution of real data but does not directly obtain real data, and then allows the cloud model to guide the edge model, so that the edge model can better analyze driving behavior, reflect the actual driving behavior, and serve as a warning and review for the driver.

[0058] Specifically, a driving style recognition method for protecting driver privacy includes:

[0059] Through data preprocessing, valid driving data is filtered out from the edge collection machine, processed into windows of the same size as needed, and this batch of labeled data is placed on the edge model to train the edge model and obtain a probability distribution. A probability distribution integration framework is constructed to calculate a global probability distribution based on the probability distribution given by each edge. The cloud model training is guided by the obtained global probability distribution, because the cloud model cannot have labeled data. After the cloud model training is completed, the probability distribution is obtained based on a batch of unlabeled driving data, and the edge model training is guided by this probability distribution. Finally, the trained model is put into the driving style recognition task of new data to complete the driving style recognition.

[0060] The driver's driving data includes: vehicle ID, timestamp, speed and acceleration on the xyz axis, longitude and latitude, road condition information, etc.

[0061] In a driving style recognition scenario, the set D = {d1, d2, ..., d N}, each driver contains a set of M frames of driving data F = {f1, f2, ..., f M}, each frame of data contains K driving data features. Each driver corresponds to a batch of data, and the relationship between the driver and the driving data features is represented by the Q matrix, Q∈R M×K .

[0062] First, the driving data is cleaned by data preprocessing to select valid data, and then the cleaned data is subjected to window partitioning operation, with a window length of w and a sliding step size of The window set is defined as E = {e1, e2, ..., e I}, I represents the number of windows after division, and the window is defined as e i ∈R w×K .

[0063] Put the obtained window into the corresponding edge model and train the edge model (taking the LSTM model as an example):

[0064] First, the driving data window is used as the input of the LSTM model, given the input feature E∈R I×w×K , the output of the LSTM layer is H. The fully connected layer converts the output of the LSTM layer into a probability distribution of categories:

[0065] H t , (h t , c t )=LSTM(E)

[0066]

[0067] Where W and b are the weights and biases of the fully connected layer, c t is the cell state at time t, h last is the hidden state of the last time step of the LSTM.

[0068] The loss function used is cross entropy loss, and is weighted to deal with the problem of class imbalance. Given the probability distribution of the model output And the true label y, the cross entropy loss L is:

[0069]

[0070] Where U is the number of samples in the test set, C is the number of categories, and y i , c is the true label of sample i belonging to category c, which is a one-hot vector, The probability that sample i is predicted to be of class c.

[0071] After training, the edge model gives the driving style recognition results of this batch of driving data. The edge calculation method is:

[0072] Accuracy is calculated for each class by counting only those cases where the predicted value is the same as the true value and dividing by the number of true values:

[0073]

[0074] Where Ac[i] is the accuracy of driving style i, N is the number of categories, i is the driving style category, and y true，j is the true value of the jth class, y pred，j is the predicted value of the jth class, ∧ represents the AND operation, which is used to connect two values. The entire expression is true only when both values ​​are i. 1(·) is an indicator function, which takes the value 1 when the condition is true and 0 otherwise;

[0075] Calculate the few-shot weight w[i]:

[0076]

[0077] Where counts[i] represents the number of samples of driving style i, total represents the total number of samples, and hum represents the number of driving styles. When counts[i] is less than the threshold of the minority class, that is, the total number of samples divided by 2 times the number of driving styles, the weight of class i is adjusted to weights, otherwise the weight is adjusted to 0.5.

[0078] Calculate the weighted accuracy wacc:

[0079]

[0080] Multiply the accuracy of each category by the corresponding category weight, add the products and divide by the weight sum to get the final single edge probability distribution.

[0081] Based on the probability distribution of each edge, the integration module calculates a probability distribution that can summarize multiple edges, including:

[0082] The model quality score qs of each edge model is calculated by multiplying the data size and overall accuracy of each edge as well as the weighted category accuracy with the corresponding weight:

[0083] qs=W d ·ds+w c ·wacc

[0084] Among them, ds is the data size, wacc is the weighted driving style fine-grained recognition accuracy, and W d is the data volume weight, w ais the overall accuracy weight, w c is the classification accuracy weight.

[0085] Use each edge probability distribution p combined with the corresponding model quality score qs to calculate the final probability distribution cp:

[0086]

[0087] Normalize the final probability to get cp' so that the sum is one:

[0088]

[0089] Next, construct the cloud model.

[0090] 1) First, construct a convolutional neural network (CNN) feature extractor:

[0091] A three-layer convolutional neural network is designed to extract one-dimensional sequence features of input data.

[0092] The first convolution layer has the input channel of the driving data dimension, the output channel is 32, the convolution kernel size is 3, and the padding is 1. It is followed by batch normalization and ReLU activation function, and a maximum pooling layer (pooling kernel size is 2 and stride is 2).

[0093] The second convolution layer has 32 input channels, 64 output channels, kernel size of 3, and padding of 1. Batch normalization and ReLU activation function are used, and the maximum pooling layer is also set.

[0094] The third convolution layer has 64 input channels, 128 output channels, a kernel size of 3, and padding of 1. Batch normalization and ReLU activation function, and finally an adaptive average pooling layer (output size is 1).

[0095] 2) Secondly, construct a bidirectional long short-term memory network (BiLSTM) time series processor:

[0096] Reshape the CNN output to meet the input requirements of BiLSTM (reshape to (bs, sl, is)).

[0097] Where bs is the batch size, that is, the number of samples processed by the model in one forward propagation, sl represents the length of the sequence, and is represents the dimension of the input feature.

[0098] Design a bidirectional LSTM layer with an input size of 128 and a hidden layer size of 64. Set 2 layers of LSTM and use bidirectional processing.

[0099] 3) Finally, construct the autoencoder embedding learning module:

[0100] Design an autoencoder module to learn the embedding representation of input data. The module contains two fully connected layers with input size of 128 and output sizes of es / / 2 and es respectively. Each fully connected layer is followed by a ReLU activation function.

[0101] es is the dimension of the embedding space, which indicates the size of the embedding vector. es / / 2 is used to specify the dimension of the internal layer of the autoencoder, especially the bottleneck layer of the autoencoder.

[0102] After constructing the cloud model, the global probability distribution cp′ is used to guide the cloud model training. After the cloud model training, it is distilled to the edge model. Specifically, the cloud model obtains a probability distribution for a batch of unlabeled data, and this probability distribution is used to guide the edge model training. The probability distribution obtained by integrating the edge model to guide the cloud model training is a distillation operation.

[0103] Input B′∈R I ×w× K Processed by convolution layer, batch normalization layer, ReLU activation function and pooling layer, B′ represents the window set;

[0104] h1=ReLU(BatchNorm(Conv1d(B′)))

[0105] h2=MaxPool(h1)

[0106] h3=ReLU(BatchNorm(Conv1d(h2)))

[0107] h4=MaxPool(h3)

[0108] h5=ReLU(BatchNorm(Conv1d(h4)))

[0109] h6=AdaptiveAvgPool(h5)

[0110] Adjust the dimensions of h6 to be suitable for BiLSTM input:

[0111] h6=Permute(h6)

[0112] Input the adjusted h6 into the BiLSTM layer:

[0113] h7, (h n , c n )=BiLSTM(h6)

[0114] Classifier head: Input the output of BiLSTM into the fully connected layer to obtain the classification result:

[0115]

[0116] Autoencoder part: Input the output of BiLSTM into the autoencoder:

[0117] h enc =ReLU(Linear(h7))

[0118] h dec =ReLU(Linear(h enc ))

[0119] h enc and h dec are the outputs of the encoder and decoder respectively.

[0120] Input B′∈R I×w×K Through the processing of convolution layer, batch normalization layer, ReLU activation function and pooling layer, the classification result and decoder output are obtained; the classification result is used to calculate the distillation loss, and the decoder output is used to calculate the reconstruction loss.

[0121] According to the global probability distribution cp′ of the edge model and the driving style recognition result P of the cloud model cloud Use the KL divergence formula to calculate the distillation loss L distill :

[0122] Calculate the loss:

[0123]

[0124] Among them, cp' i and are the probabilities corresponding to the i-th class on the edge and in the cloud, respectively, scaled using the temperature parameter T;

[0125] Use gradient descent to update the parameters of the cloud model:

[0126]

[0127] η is the learning rate, Represents the cloud model parameter θ cloud The gradient of the model is used to update the parameters of the model so that the loss function is gradually reduced, thereby optimizing the performance of the model.

[0128] Reconstructing on the feature vector rather than on the generalized driving style representation is crucial to learning a better generalized driving style representation. To minimize the loss between the reconstructed data and the original input, we use the mean squared error (MSE) to measure this loss:

[0129]

[0130] The overall minimization objective is defined as the combination of the reconstruction objective and the classification objective. For this autoencoder-based regularization with supervised learning, the unified loss function L u becomes:

[0131] L u =L distill +L r

[0132] After the cloud model is trained, driving style recognition is performed based on the unlabeled data obtained by the edge device. The results are used to guide edge model training, that is, to distill the knowledge obtained by the cloud model into the edge model, including:

[0133] According to the probability distribution P of the cloud model cloud And the edge model driving style recognition results Use the KL divergence formula to calculate the distillation loss of the i-th edge

[0134]

[0135] is the probability of the jth class in the cloud, is the probability of the jth category of the ith edge.

[0136] Use gradient descent to update the parameters of the i-th edge model:

[0137]

[0138] After the edge model is distilled, new data can be obtained from the edge device to perform the driving style recognition task.

[0139] Compared with the prior art, the present invention has the following advantages:

[0140] The above-mentioned scheme provided by the present invention extracts effective driving data from the edge device and processes it into windows of the same size through preprocessing and screening, effectively solving the problems of incomplete data and data noise. By reducing the dimension of data with too many features, the training efficiency and data quality of the model are further improved. In addition, the labeled data is trained by multiple edge models, the probability distribution of each edge is calculated, and a probability distribution integration framework is constructed based on comprehensive evaluation of factors such as data volume, overall accuracy, and weighted category accuracy. The framework effectively integrates the prediction results of different edge models to obtain a more accurate global probability distribution. Finally, this method designs a three-layer convolutional neural network to extract the one-dimensional sequence features of the input data, and adjusts its output to meet the input requirements of the bidirectional long short-term memory network (BiLSTM), making full use of the advantages of convolutional neural networks in local feature extraction and BiLSTM in time series processing, and improving the model's recognition accuracy of driving style. Finally, accurate recognition of driving style is achieved through a more comprehensive global probability distribution and a superior cloud-edge collaborative model. BRIEF DESCRIPTION OF THE DRAWINGS

[0141] Figure 1 is a flow chart of the method proposed by the present invention;

[0142] Figure 2 It is the overall model architecture diagram designed by the present invention;

[0143] Figure 3 It is a cloud model architecture diagram designed by the present invention. DETAILED DESCRIPTION

[0144] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0145] The objective of the present invention is achieved through the following technical solutions:

[0146] like Figure 1 As shown, a cloud-edge-device collaborative driving style recognition method based on privacy protection specifically includes:

[0147] Step 1: Obtain driving data and preprocess the data:

[0148] The driver's driving data includes: vehicle ID, timestamp, speed and acceleration on the xyz axis, longitude and latitude, road condition information, etc.

[0149] In a driving style recognition system, a set D = {d1, d2, ..., d N}, each driver contains a set of M frames of driving data F = {f1, f2, ..., f M}, each frame of data contains K driving data features. Each driver corresponds to a batch of data, and the relationship between the driver and the driving data features is represented by the Q matrix, Q∈R M×K .

[0150] First, the driving data is cleaned by data preprocessing to select valid data, and then the cleaned data is subjected to window partitioning operation, with a window length of w and a sliding step size of The window set is defined as E = {e1, e2, ..., e I}, I represents the number of windows after division, and the window is defined as e i ∈R w×K .

[0151] Put the obtained window into the corresponding edge model and train the edge model (taking the LSTM model as an example):

[0152] First, the driving data window is used as the input of the LSTM model, given the input feature E∈R I×w×K , the output of the LSTM layer is H. The fully connected layer converts the output of the LSTM layer into a probability distribution of categories:

[0153] H t , (h t , c t )=LSTM(E)

[0154]

[0155] Where W and b are the weights and biases of the fully connected layer, c t is the cell state at time t, h last is the hidden state of the last time step of the LSTM.

[0156] The loss function used is cross entropy loss, and is weighted to deal with the problem of class imbalance. Given the probability distribution of the model output And the true label y, the cross entropy loss L is:

[0157]

[0158] Where U is the number of samples in the test set, C is the number of categories, and y i，c is the true label of sample i belonging to category c, which is a one-hot vector. The probability that sample i is predicted to be of class c.

[0159] By calculating the loss and continuously backpropagating, the model parameters are iteratively updated to complete the training.

[0160] After training, the edge model gives the driving style recognition results of this batch of driving data. The edge calculation method is:

[0161] Accuracy is calculated for each class by counting only those cases where the predicted value is the same as the true value and dividing by the number of true values:

[0162]

[0163] Where Ac[i] is the accuracy of driving style i, S is the number of samples, i is the driving style category, and y true，j is the true value of the jth sample, y pred，j is the predicted value of the jth sample, ∧ represents the AND operation, which is used to connect two values. The entire expression is true only when both values ​​are i. 1(·) is an indicator function, which takes the value 1 when the condition is true, otherwise it takes the value 0;

[0164] Calculate the few-shot weight w[i]:

[0165]

[0166] Where counts[i] represents the number of samples of driving style i, total represents the total number of samples, and num represents the number of driving styles. When counts[i] is less than the threshold of the few-sample category, that is, the total number of samples divided by 2 times the number of driving styles, the weight of category i is adjusted to α, otherwise the weight is adjusted to 0.5.

[0167] Calculate the weighted accuracy wacc:

[0168]

[0169] Multiply the accuracy of each category by the corresponding category weight, add the products and divide by the weight sum to get the final weighted accuracy of each edge.

[0170] Based on the probability distribution of each edge, the integration module calculates a probability distribution that can summarize multiple edges, including:

[0171] The model quality score qs of each edge model is calculated by multiplying the data size and overall accuracy of each edge as well as the weighted category accuracy with the corresponding weight:

[0172] qs=w d ·ds+w c ·wacc

[0173] Among them, ds is the data size, wacc is the weighted driving style fine-grained recognition accuracy, and W d is the data volume weight, w c is the classification accuracy weight.

[0174] Use each edge probability distribution p combined with the corresponding model quality score qs to calculate the final probability distribution cp:

[0175]

[0176] N represents the number of edges, p k and qs k Represents the probability distribution and model quality score corresponding to the kth edge.

[0177] The obtained global probability distribution is normalized to obtain cp′ so that the sum is one:

[0178]

[0179] Next, construct the cloud model.

[0180] 1) First, construct a convolutional neural network (CNN) feature extractor:

[0181] A three-layer convolutional neural network is designed to extract one-dimensional sequence features of input data.

[0182] The first convolution layer has the input channel of the driving data dimension, the output channel is 32, the convolution kernel size is 3, and the padding is 1. It is followed by batch normalization and ReLU activation function, and a maximum pooling layer (pooling kernel size is 2 and stride is 2).

[0183] The second convolution layer has 32 input channels, 64 output channels, kernel size of 3, and padding of 1. Batch normalization and ReLU activation function are used, and the maximum pooling layer is also set.

[0184] The third convolution layer has 64 input channels, 128 output channels, a kernel size of 3, and padding of 1. Batch normalization and ReLU activation function, and finally an adaptive average pooling layer (output size is 1).

[0185] 2) Secondly, construct a bidirectional long short-term memory network (BiLSTM) time series processor:

[0186] The CNN output is reshaped to fit the BiLSTM input requirements (the shape is adjusted to (bs, sl, is)).

[0187] Where bs is the batch size, that is, the number of samples processed by the model in one forward propagation, sl represents the length of the sequence, and is represents the dimension of the input feature.

[0188] Design a bidirectional LSTM layer with an input size of 128 and a hidden layer size of 64. Set 2 layers of LSTM and use bidirectional processing.

[0189] 3) Finally, construct the autoencoder embedding learning module:

[0190] Design an autoencoder module to learn the embedding representation of input data. The module contains two fully connected layers with input size of 128 and output sizes of es / / 2 and es respectively. Each fully connected layer is followed by a ReLU activation function.

[0191] es is the dimension of the embedding space, which indicates the size of the embedding vector. es / / 2 is used to specify the dimension of the internal layer of the autoencoder, especially the bottleneck layer of the autoencoder.

[0192] After constructing the cloud model, the cloud model training is guided by the obtained global probability distribution cp′, and the cloud model training is distilled to the edge model. Specifically, the cloud model obtains the probability distribution of a batch of unlabeled data, and uses this probability distribution to guide the edge model training. The details are as follows:

[0193] The probability distribution obtained by edge model integration guides the cloud model training, which is a distillation operation. Specifically, it includes:

[0194] Input B′∈R I ×w× K Processed through convolutional layers, batch normalization layers, ReLU activation functions, and pooling layers:

[0195] h1=ReLU(BatchNorm(Conv1d(B)))

[0196] h2=MaxPo0l(h1)

[0197] h3=ReLU(BatchNorm(Conv1d(h2)))

[0198] h4=MaxPo0l(h3)

[0199] h5=ReLU(BatchNorm(Conv1d(h4)))

[0200] h6=AdaptiveAvgPool(h5)

[0201] Adjust the dimensions of h6 to be suitable for BiLSTM input:

[0202] h6=Permute(h6)

[0203] Input the adjusted h6 into the BiLSTM layer:

[0204] h7=BiLSTM(h6)

[0205] Classifier head: Input the output of BiLSTM into the fully connected layer to obtain the classification result:

[0206]

[0207] Autoencoder part: Input the output of BiLSTM into the autoencoder:

[0208] h enc =ReLU(Linear(h7))

[0209] h dec =ReLU(Linear(h enc ))

[0210] h enc and h dec are the outputs of the encoder and decoder respectively.

[0211] According to the global probability distribution cp′ of the edge model and the driving style recognition result P of the cloud model cloud Use the KL divergence formula to calculate the distillation loss L distill :

[0212] Calculate the loss:

[0213]

[0214] Among them, cp′ i and are the probabilities corresponding to the i-th class on the edge and in the cloud, respectively, scaled using the temperature parameter T;

[0215] Reconstructing on the feature vector rather than on the generalized driving style representation is crucial to learning a better generalized driving style representation. To minimize the loss between the reconstructed data and the original input, we use the mean squared error (MSE) to measure this loss:

[0216]

[0217] Here is the actual reconstruction vector With the original vector h dec Substitute into the formula, where is the original formula.

[0218] The overall minimization objective is defined as the combination of the reconstruction objective and the classification objective. For this autoencoder-based regularization with supervised learning, the unified loss function Lu becomes:

[0219] L u =L distill +L r

[0220] Use gradient descent to update the parameters of the cloud model:

[0221]

[0222] η is the learning rate, Represents the cloud model parameter θ cloud The gradient of the model is used to update the parameters of the model so that the loss function is gradually reduced, thereby optimizing the performance of the model.

[0223] After the cloud model is trained, driving style recognition is performed based on the unlabeled data obtained by the edge device. The results are used to guide edge model training, that is, to distill the knowledge obtained by the cloud model into the edge model, including:

[0224] According to the probability distribution P of the cloud model cloud And the edge model driving style recognition results Use the KL divergence formula to calculate the distillation loss of the i-th edge

[0225]

[0226] is the probability of the jth class in the cloud, is the probability of the jth category of the ith edge.

[0227] Use gradient descent to update the parameters of the i-th edge model:

[0228]

[0229] After the edge model is distilled, new data can be obtained from the edge device to complete the driving style recognition task.

Claims

1. A privacy-preserving cloud-edge collaborative driving style recognition method, characterized in that: The following steps are involved: 1) Preprocessing: Collect driving data on each edge device and process the driving data, including extracting features and dividing windows, normalizing the driving data, dividing the training set and the test set, and obtaining driving data feature segments and driving style categories; 2) Build an edge architecture and train an edge model. Use a neural network model on the edge to perform data enhancement operations on the data based on the driving data feature segments and driving style categories obtained in step 1), and train the edge model based on the processed driving data feature segments; 3) Build a cloud model and propose a model architecture ACB that combines convolutional neural network CNN, bidirectional long short-term memory network BiLSTM and autoencoder Autoencoder for privacy-preserving driving style recognition; 4) Build a cloud-edge collaborative framework to simulate unlabeled driving data; use the edge model to predict the classification results, and integrate the category probability distribution predicted by the edge into the cloud based on the edge model quality score. That is, use the global probability distribution to guide the cloud model training and realize the distillation of the knowledge obtained by the edge model to the cloud. Among them, unlabeled driving data refers to the driving data feature segments without driving style; 5) After the cloud model training is completed, the cloud model is deployed to the edge. The driving style recognition result is obtained on the cloud model based on the unlabeled driving data obtained on the edge. The knowledge obtained by the cloud model is distilled to the edge model, and the cloud model is used to guide the training of the edge model. Finally, the distilled edge model is used for new driving data to complete driving style recognition.

2. The privacy-preserving cloud-edge-device collaborative driving style recognition method according to claim 1, characterized in that: In step 1), the driving data features include vehicle ID, timestamp, longitude and latitude, driving speed features, driving acceleration features and road condition features; In a driving style recognition scenario, the set D of N drivers is {d1, d2, ..., d N }, each driver contains a set of M frames of driving data F = {f1, f2, ..., f M }, each frame of data contains K driving data features; the relationship between the driver and the driving data features is represented by the Q matrix, Q∈R M×K , R represents a matrix; First, the driving data is cleaned by data preprocessing, and then the cleaned data is subjected to window partitioning operation, with a window length of w and a sliding step size of The window set is defined as E = {e1, e2, ..., e I }, I represents the number of windows after division, and the window is defined as e i ∈R w×K .

3. The privacy-preserving cloud-edge-device collaborative driving style recognition method according to claim 2, characterized in that: The specific process of step 2) is as follows: Put the obtained window into the corresponding edge model and train the edge model; Take the LSTM model as an example: First, the driving data window is used as the input of the LSTM model, and the input feature E∈R is given I× w ×K , the output of the LSTM layer is H; the fully connected layer converts the output of the LSTM layer into the probability distribution of the category: H t ,(h t ,c t )=LSTM(E) Where W and b are the weights and biases of the fully connected layer, c t is the cell state at time t, h last is the hidden state of the last time step of LSTM, is the hidden state of LSTM at the tth time step, H t is the output of the LSTM layer at time t; The loss function uses the cross entropy loss and weights it, given the probability distribution of the model output And the true label y, the cross entropy loss L is: Where U is the number of samples in the test set, C is the number of categories, and y i，c is the true label of sample i belonging to category c, which is a one-hot vector. The probability that sample i is predicted to be of class c.

4. The privacy-preserving cloud-edge collaborative driving style recognition method according to claim 1, characterized in that: The cloud model building process in step 3) is as follows: 3.1) Construct a convolutional neural network CNN feature extractor: Design a three-layer convolutional neural network to extract the one-dimensional sequence features of the input data; The first convolution layer: the input channel is the driving data dimension, the output channel is 32, the convolution kernel size is 3, and the padding is 1; followed by batch normalization and ReLU activation function, and a maximum pooling layer with a pooling kernel size of 2 and a stride of 2; The second convolution layer has 32 input channels, 64 output channels, a kernel size of 3, and a padding of 1. This is followed by batch normalization and ReLU activation, and the maximum pooling layer is set the same as above. The third convolution layer: input channels are 64, output channels are 128, convolution kernel size is 3, padding is 1; batch normalization and ReLU activation function, and finally an adaptive average pooling layer with an output size of 1; 3.2) Construct a bidirectional long short-term memory network BiLSTM time series processor: Reshape the CNN output to meet the input requirements of BiLSTM, and adjust the shape to (bs, sl, is); Where bs is the batch size, that is, the number of samples processed by the model in one forward propagation, sl represents the length of the sequence, and is represents the dimension of the input feature; Design a bidirectional LSTM layer with an input size of 128 and a hidden layer size of 64. Set 2 layers of LSTM and use bidirectional processing. 3.3) Construct an autoencoder embedding learning module: Design an autoencoder module to learn the embedded representation of input data; the module contains two fully connected layers with an input size of 128 and output sizes of es / / 2 and es respectively; each fully connected layer is followed by a ReLU activation function; es is the dimension of the embedding space, indicating the size of the embedding vector; es / / 2 is used to specify the dimension of the internal layer of the autoencoder.

5. The privacy-preserving cloud-edge collaborative driving style recognition method according to claim 1, characterized in that: In step 4), the cloud-edge-end collaborative architecture includes a distillation branch from the edge-end to the cloud-end and a distillation branch from the cloud-end to the edge-end; In edge-to-cloud distillation, we first use the simulated data on the cloud to make all edges derive corresponding probability distributions, including: Calculate the accuracy for each category separately, count the number of predicted values ​​that are the same as the true value, and then divide it by the number of true values: Where Ac[i] is the accuracy of driving style i, N is the number of categories, i is the driving style category, and y true，j is the true value of the jth class, y pred,j is the predicted value of the jth class, ∧ represents the AND operation, which is used to connect two values. The entire expression is true only when both values ​​are i; 1(·) is an indicator function, which takes the value 1 when the condition is true and 0 otherwise; Calculate the weight w[i] of driving style i: Where counts[i] represents the number of samples of driving style i, total represents the total number of samples, and num represents the number of driving styles. When counts[i] is less than the threshold of the sample category, that is, the total number of samples divided by 2 times the number of driving styles, the weight of category i is adjusted to weights, otherwise the weight is adjusted to 0.5; Calculate the weighted accuracy wacc: Multiply the accuracy of each category by the corresponding category weight, add the products and divide by the weight sum to get the final single edge probability distribution; Based on the probability distribution of each edge, a probability distribution that can summarize multiple edges is calculated, including: The model quality score qs of each edge model is calculated by multiplying the data size and overall accuracy of each edge as well as the weighted category accuracy with the corresponding weight: qs=w d ·ds+w c ·wacc Among them, ds is the data size, wacc is the weighted driving style fine-grained recognition accuracy, and w d is the data volume weight, w a is the overall accuracy weight, w c is the classification accuracy weight; Use each edge probability distribution p combined with the corresponding model quality score qs to calculate the final probability distribution cp: Normalize the final probability to get cp′ so that the sum is one: The probability distribution obtained by edge model integration guides cloud model training, including: According to the global probability distribution cp′ of the edge model and the driving style recognition result P of the cloud model cloud Use the KL divergence formula to calculate the distillation loss L distill : Calculate the loss: Among them, cp′ i and are the probabilities corresponding to the i-th class on the edge and in the cloud, respectively, scaled using the temperature parameter T; The reconstruction loss is measured using mean squared error (MSE): in is the reconstructed vector, z is the original vector; The overall minimization objective is defined as the combination of the reconstruction objective and the classification objective; for autoencoder-based regularization with supervised learning, the unified loss function L u becomes: L u =L distill +L r Use gradient descent to update the parameters θ of the cloud model cloud : η is the learning rate, Represents the cloud model parameter θ cloud The gradient of the model is used to update the model parameters so that the loss function is gradually reduced, thereby optimizing the performance of the model.

6. The privacy-preserving cloud-edge collaborative driving style recognition method according to claim 1, characterized in that: In step 5), the knowledge acquired by the cloud model is distilled to the edge model, which includes: According to the probability distribution P of the cloud model cloud And the edge model driving style recognition results Use the KL divergence formula to calculate the distillation loss of the i-th edge is the probability of the jth class in the cloud, is the probability of the ,th category of the ith edge; Use gradient descent to update the parameters of the i-th edge model: After the edge model is distilled, new data can be obtained from the edge device to perform the driving style recognition task.

Images