A method for identifying driver distracted behavior based on federated learning

Abstract

The present invention discloses a method for identifying driver distraction behavior based on federated learning. The application scenarios targeted by the present invention mainly include a target detection model, a server and a number of users, and each user has a mutually independent private data set. The present invention innovatively combines the federated learning algorithm with the target detection algorithm. Under the constraint that the server cannot obtain the user data set, the training of the YOLOv3 model for detecting driver distraction behavior is realized. At the same time, an innovative method for using a learning rate preheating method in the federated learning process is proposed to improve the model convergence efficiency and performance. The method of the present invention does not require too much prior knowledge, does not need to obtain the original data of each user, meets the needs of privacy protection, and the present invention has good reusability in similar application scenarios, and the practical value of the invention is relatively strong.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and specifically to a method for identifying driver distraction behavior based on federated learning. Background Art

[0002] With the introduction and continuous development of intelligent driving technology, it has been applied to some high-end cars as an auxiliary driving device. In the past decade, many methods for identifying driver distraction have been gradually proposed, including vehicle measurement, behavior measurement and physiological measurement. At the same time, with the development of machine learning, image recognition technology and target detection technology have also been gradually applied to detect driver distraction.

[0003] Generally speaking, target detection algorithms are currently divided into two-step and one-step methods. The two-step method means that the detection algorithm needs to be completed in two steps. First, the candidate area needs to be obtained, and then the classification is performed, such as the R-CNN series; in contrast, the one-step detection can be understood as a one-step method, without the need to find the candidate area separately, and typical ones are SSD / YOLO.

[0004] Normally, except for a few giant companies, most companies face the problem of small data volume or poor data quality, which is not enough to support the realization of artificial intelligence technology. Federated learning is essentially a distributed machine learning technology or machine learning framework. Its goal is to achieve joint modeling and improve the effectiveness of AI models on the basis of ensuring data privacy security and legal compliance.

[0005] In summary, the present invention proposes a method for identifying driver distraction behavior based on federated learning, which is used to detect driver distraction behavior while avoiding the problem of driver privacy leakage. Summary of the invention

[0006] The purpose of the present invention is to solve the problem of target detection and privacy protection in driver distraction behavior recognition. The application scenarios targeted by the present invention mainly include target detection models, servers and several users, and each user has a mutually independent private data set. In order to ensure the maximum utilization of the data set and privacy protection at the same time, the present invention proposes a driver distraction behavior recognition method based on federated learning.

[0007] In order to achieve the above object, the technical solution adopted by the present invention comprises the following steps:

[0008] Step 1: Create a driver behavior dataset.

[0009] In the present invention, there are K users and original data S = {S1, S2...S K}, where S kRepresents the original data owned by the kth user, and the original data owned by each user are independent of each other.

[0010] For each user’s original data, labelImg software is used to label each target in each image in the original data to obtain samples, and these samples are used to automatically generate the preprocessed data set D = {D1, D2...D K Each sample consists of the labeled image and the location information of the target in the image, including the horizontal center coordinate x, vertical center coordinate y, width w, and height h of the target prediction box. At the same time, each preprocessed data set is divided into a training set, a validation set, and a test set according to a certain ratio.

[0011] Step 2: Build a federated learning model and set federated learning parameters.

[0012] In the present invention, there is a server and a target detection model. The global target detection model in the server and the local target detection model of the user have the same framework. Among them, the parameters of the global target detection model are initialized by loading the official pre-trained weights.

[0013] The server will set the maximum federated learning training round T, the termination training threshold M, the number of epochs for the user's local object detection model training in a single round of federated learning, and the learning rate α. The learning rate α is set by the server in each federated learning training round and is updated by learning rate warm-up to make the model converge faster and achieve better results.

[0014] Step 3: Training of local object detection model.

[0015] Each user obtains the parameters of the global target detection model from the server and loads them into their own local target detection model. They use their own training set and validation set to train the local target detection model and calculate the loss. They use the gradient descent method to calculate the gradient. After reaching the training epoch, they upload the obtained gradient to the server.

[0016] Step 4: Update the server global target detection model.

[0017] After the server obtains the gradients uploaded by all K users, it updates the parameters of the global object detection model. The specific formula is as follows:

[0018]

[0019] Among them, θ t+1 is the updated global target detection model parameter, θ t is the current global target detection model parameter, n kis the number of training set samples for the kth user, n is the sum of the number of training set samples for K users, and w k Gradient uploaded by the kth user.

[0020] Step 5: Global object detection model evaluation.

[0021] After the global object detection model is updated, the updated parameters are passed to the user's local object detection model. The user uses the test set to test the performance of the global object detection model, using the average accuracy mAP of all categories to measure. If the training rounds reach the maximum rounds or the mAP value reaches the termination training threshold M, the federated learning training is terminated, otherwise repeat steps 3 and 4.

[0022] Step 6: Identify driver distraction behavior.

[0023] Driver distraction behavior recognition: Use the final global object detection model to perform object detection on the driver's image to determine whether the driver is distracted.

[0024] Beneficial effects of the present invention:

[0025] The present invention is based on a commonly used target detection model, and improves the problem of driver privacy protection. A federated learning method is added to the original model to protect personal privacy. The present invention innovatively combines the federated learning algorithm with the target detection algorithm. Under the constraint that the server cannot obtain the user data set, the training of the YOLOv3 model for detecting driver distraction behavior is realized. At the same time, an innovative method of using learning rate preheating in the federated learning process is proposed to improve the model convergence efficiency and performance. The method of the present invention does not require too much prior knowledge, does not need to obtain the original data of each user, meets the needs of privacy protection, and the present invention has good reusability in similar application scenarios, and the practical value of the invention is relatively strong. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flowchart of federated learning provided by an embodiment of the present invention;

[0027] Figure 2 It is a diagram of the federated learning structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] like Figure 1 As shown, a method for identifying driver distraction behavior based on federated learning includes the following steps:

[0030] Step 1: Create a driver behavior dataset. The user collects driver behavior images and annotates the original images to obtain a preprocessed driver behavior dataset. The dataset is divided into a training set, a validation set, and a test set according to a certain ratio.

[0031] Step 2: Build a federated learning model and set federated learning parameters. The global target detection model in the server and the user's local target detection model have the same framework. The parameters of the global target detection model will be initialized by loading the official pre-trained weights. At the same time, set the maximum rounds of the federated learning method, the termination training threshold, etc.

[0032] Step 3: Set the learning rate. Using the learning rate warm-up method, the server sets the learning rate in each federated learning round.

[0033] Step 4: Training of the local target detection model. Each user obtains the parameters of the global target detection model from the server and loads them into their own local target detection model. The local target detection model is trained using their own training set and validation set and the loss is calculated. The gradient is calculated using the gradient descent method. After reaching the training epoch, the obtained gradient is uploaded to the server.

[0034] Step 5: Update the global target detection model of the server. After receiving the gradients uploaded by each user, the server performs weighted summation of each gradient through the federated learning algorithm and updates the parameters of the global target detection model.

[0035] Step 6: Global target detection model evaluation performance test; after the global target detection model is updated, the updated parameters are passed to the user's local target detection model. The user uses the test set to test the performance of the global target detection model, and uses the average accuracy (mAP) of all categories to measure it; if the number of training rounds reaches the maximum round or the mAP value reaches the termination training threshold M, the federated learning training is terminated, otherwise repeat steps 4 and 5;

[0036] Step 7: Use the local target detection model finally obtained to perform target recognition, and use the local target detection model to identify the driver's distraction behavior.

[0037] Furthermore, the preparation of the driver behavior dataset described in step 1 is specifically implemented as follows:

[0038] In the present invention, there are K users and original data S = {S1, S2...S K}, where S k Represents the original data owned by the kth user, and the original data owned by each user are independent of each other.

[0039] For each user’s original data, labelImg software is used to label each target in each image in the original data to obtain samples, and these samples are used to automatically generate the preprocessed data set D = {D1, D2...D K Each sample consists of the labeled image and the location information of the target in the image, including the horizontal center coordinate x, vertical center coordinate y, width w, and height h of the target prediction box. At the same time, each preprocessed data set is divided into a training set, a validation set, and a test set according to a certain ratio.

[0040] Furthermore, the construction of the federated learning model and setting of the federated learning parameters described in step 2 are specifically implemented as follows:

[0041] In the present invention, there is a server and a target detection model. The global target detection model in the server and the local target detection model of the user have the same framework. Among them, the parameters of the global target detection model are initialized by loading the official pre-trained weights.

[0042] The server will set the maximum federated learning training round T, the termination training threshold M, the number of epochs for the user's local object detection model training in a single round of federated learning, and the learning rate α. The learning rate α is set by the server in each federated learning training round and is updated by learning rate warm-up to make the model converge faster and achieve better results.

[0043] Furthermore, the training of the local target detection model described in step 4 is specifically implemented as follows:

[0044] Each user obtains the parameters of the global target detection model from the server and loads them into their own local target detection model. They use their own training set and validation set to train the local target detection model and calculate the loss. They use the gradient descent method to calculate the gradient. After reaching the training epoch, they upload the obtained gradient to the server.

[0045] Furthermore, the server global object detection model update described in step 5 is specifically implemented as follows:

[0046] like Figure 2 As shown in the figure, after the server obtains the gradients uploaded by all K users, it updates the parameters of the global object detection model. The specific formula is as follows:

[0047]

[0048] Among them, θ t+1 is the updated global target detection model parameter, θ t is the current global target detection model parameter, n kis the number of training set samples for the kth user, n is the sum of the number of training set samples for K users, and w k Gradient uploaded by the kth user.

[0049] Furthermore, the global object detection model evaluation described in step 6 is specifically implemented as follows:

[0050] After the global object detection model is updated, the updated parameters are passed to the user's local object detection model. The user uses the test set to test the performance of the global object detection model, using the average accuracy mAP of all categories to measure. If the training rounds reach the maximum rounds or the mAP value reaches the termination training threshold M, the federated learning training is terminated, otherwise repeat steps 3 and 4.

[0051] Furthermore, the driver distraction behavior identification described in step 7 is specifically implemented as follows:

[0052] The final local object detection model is used to perform object detection on the driver's image to determine whether the driver is distracted.

[0053] It should be known that the parts not elaborated in detail in this specification belong to the prior art. Relevant technical personnel should understand that the above embodiments are only to help readers understand the principles and implementation methods of the present invention, and the scope of protection of the present invention is not limited to such embodiments. All equivalent replacements made on the basis of the present invention are within the protection scope of the rights of the present invention.

Claims

1. A method for identifying driver distraction behavior based on federated learning, characterized in that The following steps are involved: Step 1: Create a driver behavior dataset; The user collects driver behavior images and annotates the original images to obtain a preprocessed driver behavior dataset; Divide the data set into training set, validation set and test set according to a certain ratio; Step 2: Build a federated learning model and set federated learning parameters. The global target detection model in the server and the user's local target detection model have the same framework. The parameters of the global target detection model will be initialized by loading the official pre-trained weights. At the same time, the maximum rounds and termination training threshold of the federated learning method are set. Step 3: Set the learning rate. Using the learning rate warm-up method, the server sets the learning rate in each federated learning round. Step 4: Training of the local target detection model. Each user obtains the parameters of the global target detection model from the server and loads them into their own local target detection model. The local target detection model is trained using their own training set and validation set and the loss is calculated. The gradient is calculated using the gradient descent method. After reaching the training epoch, the obtained gradient is uploaded to the server. Step 5: Update the global target detection model of the server. After receiving the gradients uploaded by each user, the server performs weighted summation of each gradient through the federated learning algorithm and updates the parameters of the global target detection model. Step 6: Global target detection model evaluation performance test; after the global target detection model is updated, the updated parameters are passed to the user's local target detection model. The user uses the test set to test the performance of the global target detection model, and uses the average accuracy (mAP) of all categories to measure it; if the number of training rounds reaches the maximum round or the mAP value reaches the termination training threshold M, the federated learning training is terminated, otherwise repeat steps 4 and 5; Step 7: Use the local target detection model finally obtained to perform target recognition, and use the local target detection model to identify the driver's distraction behavior; Step 5 is implemented as follows: After the server obtains the gradients uploaded by all K users, it updates the parameters of the global object detection model. The specific formula is as follows: Among them, θ t+1 is the updated global target detection model parameter, θ t is the current global target detection model parameter, n k is the number of training set samples for the kth user, n is the sum of the number of training set samples for K users, and w k Gradient uploaded by the kth user.

2. The method for identifying driver distraction behavior based on federated learning according to claim 1, characterized in that Step 1 is implemented as follows: There are K users and original data S = {S1, S2...S K }, where S k Represents the original data owned by the kth user, and the original data owned by each user is independent of each other; For each user’s original data, labelImg software is used to label each target in each image in the original data to obtain samples, and these samples are used to automatically generate the preprocessed data set D = {D1, D2...D K }; Each sample consists of the labeled image and the location information of the target in the image, including the horizontal center coordinate x, vertical center coordinate y, width w, and height h of the target prediction box; at the same time, each preprocessed data set is divided into training set, validation set, and test set according to a certain ratio.

Images