A driving behavior road condition warning method based on federated learning

Abstract

The present application provides a method for early warning of driving behavior and road conditions based on federated learning, including: automatically identifying the identity of the driver and collecting the driver's driving behavior information; collecting, transmitting and protecting the privacy of the driver's information during use based on a differential privacy algorithm combined with federated learning; analyzing the driver's driving habits and personality based on a deep learning method; analyzing the road hazard factors during driving based on an image processing algorithm; calculating the probability of various types of traffic accidents based on the bad driving habit data and personality portraits of surrounding drivers and road condition analysis data; and planning a driving operation plan based on the current potential probability of traffic accidents, specifically including: establishing a driving operation planning model and planning a driving operation plan.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a driving behavior road condition early warning method based on federated learning. Background Art

[0002] When a car is driving, it often encounters various traffic accidents, and the cause of the accident is usually the driver's negligence and bad driving habits. Therefore, many times, even if a driver has good driving habits, high driving skills, and is responsible, it is difficult to avoid collisions, scratches, and other traffic accidents with other unqualified drivers on the road. Therefore, if you can obtain the personality characteristics and driving habits of the drivers around you while driving, you can provide a reference for the current driving operation and avoid those drivers around you who may cause traffic friction and disputes with you. Summary of the invention

[0003] The present invention provides a driving behavior road condition warning method based on federated learning, which mainly includes:

[0004] Automatically identify the driver's identity and collect the driver's driving behavior information; collect, transmit and protect the privacy of driver information based on differential privacy algorithm combined with federated learning; analyze the driver's driving habits and personality based on deep learning methods; analyze the road hazard factors during driving based on image processing algorithms; calculate the probability of various traffic accidents based on the bad driving habits data and personality portraits of surrounding drivers and road condition analysis data; plan driving operation plans based on the current potential traffic accident probability;

[0005] Further optionally, the automatically identifying the driver's identity and collecting the driver's driving behavior information includes:

[0006] Use the in-car camera to capture the driver's facial image when the driver enters the cab, and determine the driver's identity through face recognition technology; after determining the identity, collect driving behavior information for the current driver, first obtain the driver's traffic violation penalty record by calling the traffic management bureau's query interface; then collect the driver's current various in-car activity records through the in-car camera and driving recorder; finally, save the collected traffic violation penalty records and in-car activity records to the current driver's account. If the current driver does not have an account, indicating that this is the first time using this system, then create a new account before recording the information.

[0007] Further optionally, the privacy protection during driver information collection, transmission and use based on differential privacy algorithm combined with federated learning includes:

[0008] The driver's driving behavior information and traffic violation records collected during driving are protected by differential privacy algorithm combined with federated learning; each neural network model is trained based on the federated learning method; first, the neural network model is trained locally on the client to obtain local model parameters; then, random noise is added to the local model parameters obtained after training based on the differential privacy algorithm; the server aggregates the local model parameters received from the client to obtain new global model parameters; finally, the server broadcasts the new global model parameters to each client, and the client accepts the new global model parameters and recalculates locally; including: training the neural network model based on federated learning; adding noise to the client model parameters based on the differential privacy algorithm;

[0009] The training of the neural network model based on federated learning specifically includes:

[0010] Based on the federated learning method, each neural network model is trained, and the neural network model includes: a neural network model for analyzing the driver's driving habits and personality, a road condition analysis model, a traffic accident analysis model, and a driving operation planning model. First, the server broadcasts each initial neural network model to the local client of each vehicle; then the local client trains each initial neural network model based on the driver's personal information and driving data collected locally, and feeds back the local model parameters obtained after training to the server; then the server uses the FedAVG algorithm to aggregate the local model parameters received from the local client to obtain a new global model parameter; finally, the server broadcasts the new global model parameter to each local client; the local client accepts the new global model parameter and updates each initial neural network model, re-calculates the local calculation and the next round of model iteration.

[0011] The method of adding noise to the client model parameters based on the differential privacy algorithm specifically includes:

[0012] After training the neural network model locally on the client to obtain the model parameters based on federated learning, the differential privacy algorithm is used to add random noise to the local model parameters obtained after training. For the numerical data in the local model parameters, the Laplace mechanism is used to add random dynamic disturbance noise to the data; for non-numerical, discrete data, the exponential mechanism is used to determine the probability of the output result, and the output probability is determined according to the proportion of discrete data. The higher the proportion, the higher the corresponding discrete data output probability.

[0013] Further optionally, the analysis of the driver's driving habits and personality based on the deep learning method includes:

[0014] An initial neural network model for analyzing the driver's driving habits and personality is established based on a deep learning method, and the initial neural network model includes: a bad driving habit recognition model and a driver personality analysis model; the above two models are then broadcast to the local client of the user's vehicle, and the driving-related personality characteristics and driving habits of each driver are calculated based on local data; the model parameters are then adjusted based on the calculation results, and the updated model is fed back to the server; finally, the server aggregates the local model received from the local client to obtain a new global model parameter, and broadcasts it to the local client again; including: training a bad driving habit recognition model based on traffic violation records; establishing a driver personality analysis model based on a deep learning method; and analyzing the bad driving habits and personality portraits of drivers around the vehicle in real time;

[0015] The training of the bad driving habit recognition model based on traffic violation records specifically includes:

[0016] The bad driving habit recognition model is trained based on the current driver's historical traffic violation records and deep learning methods. First, a large number of traffic violation records of various drivers are collected as training sets and test sets; then the training set is cleaned, converted and feature extracted, and the extracted features include: violation factors, penalty types, fine types, whether points are deducted, deduction value, and number of violations; then the samples are labeled and classified, and the samples are marked with various bad driving habit labels, and the bad driving habit labels include: driving in the opposite direction, running a red light, changing lanes illegally, speeding, illegal parking, crossing the line, not wearing a seat belt, drunk driving, talking on the phone while driving, smoking and deliberately covering the license plate; then the training set is input into the bad driving habit recognition model to train the neural network and establish classification decision rules; finally, the test set is input to test the accuracy of the bad driving habit recognition model, and the parameters of the bad driving habit recognition model are adjusted according to the test results to improve the accuracy of model recognition.

[0017] The driver personality analysis model is established based on the deep learning method, specifically including:

[0018] A driver personality analysis model is established, and the driver's personality profile is automatically created based on the in-car activity records and the driver's personal information to analyze the driver's driving personality characteristics. First, a driver personality analysis model is established based on a deep learning method; characteristic values ​​related to the driver's personality are extracted from the personal information of various drivers, including: gender, age, occupation, education level, driving experience and bad driving habits, and emotional keywords are extracted from the voice data in the dashcam data and in-car activity records. These are used as characteristic values ​​to input into the driver personality analysis neural network for model training, and then the driver's personality profile is output. The personality profile includes multiple dimensions: driving experience, driving preference, driving caution, driving concentration and driving patience.

[0019] The real-time analysis of the bad driving habits and personality profiles of drivers around the vehicle specifically includes:

[0020] The bad driving habit recognition model and driver personality analysis model are broadcast to the local client of each user's vehicle, and the personality profile and bad driving habits of each driver are calculated based on the current traffic violation records, in-car activity records and personal information collected locally in real time. The local client then feeds back the personality profile and bad driving habits of each driver to the server. The server calculates the spatial distribution of each vehicle on the road based on the positioning of each vehicle, and then feeds back the personality profile and bad driving habits of drivers in other vehicles that are adjacent to the current vehicle to the local client based on the spatial distribution data.

[0021] Further optionally, the analyzing the road hazard factors during driving based on the image processing algorithm includes:

[0022] Establishing a road condition analysis model based on a target recognition algorithm, identifying and classifying objects in the vehicle's field of view, and obtaining road condition analysis data for the current road; including: establishing a road condition analysis model based on a target recognition algorithm; identifying factors affecting the traffic conditions in the vehicle's surroundings in real time;

[0023] The establishment of a road condition analysis model based on a target recognition algorithm specifically includes:

[0024] First, a large number of images of the road environment outside the vehicle are collected by the on-board camera as training data, which are divided into a training set and a test set; then the training set is preprocessed to reduce invalid and interfering information in the image, and the image preprocessing includes: grayscale processing and histogram equalization. Then, the preprocessed image is segmented based on the simple linear iterative clustering SLIC algorithm, and the image is segmented into multiple superpixels according to the area occupied by various objects in the image. The superpixels generated by image segmentation are manually annotated, and the object category in the candidate frame is used as a label, and the label value is various traffic condition influencing factors, and the traffic condition influencing factors include: zebra crossings, speed bumps, traffic lights, roadblocks, wide-angle lenses, guardrails, car blocks, toll booths, traffic signs, traffic prohibition signs, traffic guide signs, traffic warning signs, road construction safety signs, tourist area signs, auxiliary signs, road bumps, water and snow, and passing pedestrians and animals. Then, the labeled superpixels are input into the CNN image classifier for training. The CNN image classifier extracts the image features of the superpixels corresponding to each label and establishes classification decision rules. Finally, the test set is input into the CNN image classifier, and the classification test is performed according to the classification decision rules established by the CNN image classifier to evaluate the accuracy of the classification results and adjust the model parameters according to the test results.

[0025] The real-time identification of factors affecting the traffic conditions of the vehicle's surrounding environment specifically includes:

[0026] The local client of the vehicle downloads the road condition analysis model from the server, and then identifies and classifies objects in the vehicle's field of view in real time based on the images collected by the on-board camera, discovers the factors affecting the traffic conditions on the current road, and counts the location, number and distance of the factors affecting the traffic conditions to obtain road condition analysis data.

[0027] Further optionally, the calculation of the probability of various types of traffic accidents based on the bad driving habit data and personality portraits of surrounding drivers and the road condition analysis data includes:

[0028] Establish a traffic accident case library, and establish a traffic accident analysis model based on the traffic accident case library; obtain the bad driving habits and personality analysis data of drivers of surrounding vehicles and the road condition analysis data of the current road in real time, input the traffic accident analysis model, and calculate various types of possible traffic accidents; including: establishing a traffic accident case library; establishing a traffic accident probability prediction model; and calculating the probability of various types of traffic accidents in real time;

[0029] The establishment of a traffic accident case database specifically includes:

[0030] First, the time, location, weather, cause of the accident, road condition images, personal information of the drivers of the vehicles involved and surrounding vehicles, traffic violation records, and in-car activity records of each traffic accident are collected; then the road condition images are input into the road condition analysis model to obtain the road condition analysis data of the traffic accident location; then the personal information and in-car activity records of the drivers of the vehicles involved and surrounding vehicles are input into the driver personality analysis model, and the traffic violation records are input into the bad driving habit recognition model to obtain the personality portraits and bad driving habit data of the drivers of the vehicles involved and surrounding vehicles. Finally, the time, location, weather, accident type, road condition analysis data of each traffic accident, as well as the personality portraits and bad driving habit data of the drivers of the vehicles involved and surrounding vehicles are entered into the traffic accident case database.

[0031] The establishment of a traffic accident probability prediction model specifically includes:

[0032] Based on the traffic accident case library and the support vector machine (SVM) classification prediction model, a traffic accident probability prediction model is constructed to analyze the probability of various types of traffic accidents. First, various types of traffic accident cases are used as samples to obtain the influencing factors of traffic accidents, including: time, location, weather, road condition analysis data, and personality portraits and bad driving habits of the drivers of the involved vehicles and surrounding vehicles; then a single-classification SVM model is constructed based on the influencing factors of traffic accidents; then the activation function is used to map the value interval of the distance from the intermediate output sample of the single-classification SVM model to the center of the hypersphere in the model to [0,1], and the mapping result is the probability of a certain type of traffic accident with a surrounding vehicle.

[0033] The real-time calculation of the probability of various types of traffic accidents specifically includes:

[0034] The vehicle loads the traffic accident probability prediction model from the server to the local computer, and collects the time, location, weather, and road condition analysis data of the current location. It then obtains the bad driving habits and personality portraits of the drivers of the surrounding vehicles from the server, inputs the traffic accident probability prediction model, and calculates in real time the probability of a certain type of traffic accident between the current vehicle and the surrounding vehicles in front, behind, left, right, and four diagonal corners. It then sorts the probabilities of various types of traffic accidents for each vehicle, generates 8 traffic accident probability tables corresponding to the front, behind, left, right, and four diagonal corners, and dynamically updates the sorting. Finally, it extracts the traffic accident probability ranked first in the 8 traffic accident probability tables, and forms a 3X3 danger matrix according to the vehicle orientation, where the element in the center of the matrix corresponds to the current vehicle, with a value of 1, and the other 8 elements correspond to vehicles in 8 orientations, and are filled with corresponding values.

[0035] Further optionally, planning a driving operation plan according to the current potential traffic accident probability includes:

[0036] A driving operation planning model is established based on deep learning methods, and several subsequent driving operation plans are planned according to the current danger matrix, and real-time feedback is given to the driver; including: establishing a driving operation planning model; planning a driving operation plan;

[0037] The establishing of the driving operation planning model specifically includes:

[0038] A driving operation planning model is established based on the deep learning method. First, the danger matrix in a large amount of user historical data is collected, and the elements of the danger matrix are extracted. The name, probability value, and element coordinates of the traffic accident in the danger matrix elements are used as features, and the standard driving operations for dealing with traffic accidents in the danger matrix elements are used as labels to train the driving operation planning model.

[0039] The planned driving operation plan specifically includes:

[0040] The server side loads the driving operation planning model to each vehicle client. The vehicle client plans the driving operation plan in real time according to the current vehicle's danger matrix data, and notifies the driver to implement the driving operation plan through voice prompts. The driving operation plan is updated in real time based on the danger matrix data and the driver's driving operation changes; when the danger matrix data is updated or the system captures the driver's driving operation, the system determines that the current node is for driving operation plan planning and re-plans the next driving operation plan.

[0041] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:

[0042] The present invention can protect privacy while obtaining information about surrounding drivers based on a differential privacy algorithm combined with federated learning, and judge the probability of surrounding traffic accidents based on road condition data, personality portraits of surrounding drivers and bad driving habits, and decide in which direction the vehicle should go to avoid various most likely traffic accident hazards. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The present invention is a flowchart of a method for early warning of driving behavior and road conditions based on federated learning.

[0044] Figure 2 It is a schematic diagram of a driving behavior road condition warning method based on federated learning of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 FIG. 1 is a flow chart of a method for warning driving behavior and road conditions based on federated learning of the present invention. Figure 1 As shown, a driving behavior road condition warning method based on federated learning in this embodiment may specifically include:

[0047] Step 101, automatically identify the driver's identity and collect the driver's driving behavior information.

[0048] The in-car camera is used to collect the driver's facial image when the driver enters the cab, and the driver's identity is determined through face recognition technology. After the identity is determined, the driving behavior information of the current driver is collected. First, the query interface of the traffic management bureau is called to obtain the driver's traffic violation penalty record; then the in-car camera and driving recorder are used to collect the driver's current various in-car activity records; finally, the collected traffic violation penalty records and in-car activity records are recorded in the current driver's account. If the current driver does not have an account, indicating that he is using this system for the first time, then a new account is created before recording the information.

[0049] Step 102: Driver information collection, transmission and privacy protection during use are performed based on a differential privacy algorithm combined with federated learning.

[0050] The privacy of the driver's driving behavior information and traffic violation records collected during driving is protected by the differential privacy algorithm combined with federated learning. Each neural network model is trained based on the federated learning method; first, the neural network model is trained locally on the client to obtain the local model parameters; then, random noise is added to the local model parameters obtained after training based on the differential privacy algorithm; the server aggregates the local model parameters received from the client to obtain the new global model parameters; finally, the server broadcasts the new global model parameters to each client, and the client accepts the new global model parameters and recalculates locally.

[0051] Training neural network models based on federated learning.

[0052] Based on the federated learning method, each neural network model is trained, and the neural network model includes: a neural network model for analyzing the driver's driving habits and personality, a road condition analysis model, a traffic accident analysis model, and a driving operation planning model. First, the server broadcasts each initial neural network model to the local client of each vehicle; then the local client trains each initial neural network model based on the local collected driver's personal information and driving data, and feeds back the local model parameters obtained after training to the server; then the server uses the FedAVG algorithm to aggregate the local model parameters received from the local client to obtain new global model parameters; finally, the server broadcasts the new global model parameters to each local client; the local client accepts the new global model parameters and updates each initial neural network model, recalculates locally and iterates the next round of models. For example: the driver's in-car activity records and traffic violation penalty records largely reflect the driver's driving habits and personality. The driver's driving habits and personality are important factors affecting driving safety. Therefore, the driver's in-car activity records and traffic violation penalty records need to be collected for analysis during driving. A car may be driven by multiple people, so the driver's identity is not fixed. When driver Zhang San enters the cab, the system performs facial recognition on him and calls the interface of the Traffic Management Bureau to confirm Zhang San's identity and obtain his traffic violation penalty record. Then the camera and driving recorder begin to record Zhang San's various activities in the car in real time. When Zhang San's colleague Li Si enters the cab, the system performs facial recognition on him and calls the interface of the Traffic Management Bureau to confirm Li Si's identity. However, Li Si is using the system for the first time and does not have an account in the system. Therefore, the system automatically creates an account for Li Si first, and then records Li Si's relevant information.

[0053] Noise is added to the client model parameters based on the differential privacy algorithm.

[0054] After training the neural network model locally on the client to obtain the model parameters based on federated learning, the differential privacy algorithm is used to add random noise to the local model parameters obtained after training. For the numerical data in the local model parameters, the Laplace mechanism is used to add random dynamic disturbance noise to the data; for non-numerical, discrete data, the exponential mechanism is used to determine the probability of the output result, and the output probability is determined according to the proportion of discrete data. The higher the proportion, the higher the corresponding discrete data output probability.

[0055] Step 103: Analyze the driver's driving habits and personality based on a deep learning method.

[0056] An initial neural network model for analyzing the driver's driving habits and personality is established based on a deep learning method, and the initial neural network model includes: a bad driving habit recognition model and a driver personality analysis model; the above two models are then broadcast to the local client of the user's vehicle, and the driving-related personality characteristics and driving habits of each driver are calculated based on local data; the model parameters are then adjusted based on the calculation results, and the updated model is fed back to the server; finally, the server aggregates the local models received from the local clients to obtain new global model parameters, which are broadcasted to the local clients again.

[0057] Train a bad driving habit recognition model based on traffic violation records.

[0058] The bad driving habit recognition model is trained based on the current driver's historical traffic violation records and deep learning methods. First, a large number of traffic violation records of various drivers are collected as training sets and test sets; then the training set is cleaned, converted and feature extracted, and the extracted features include: violation factors, penalty types, fine types, whether points are deducted, deduction value, and number of violations; then the samples are labeled and classified, and the samples are marked with various bad driving habit labels, and the bad driving habit labels include: driving in the opposite direction, running a red light, changing lanes illegally, speeding, illegal parking, crossing the line, not wearing a seat belt, drunk driving, talking on the phone while driving, smoking and deliberately covering the license plate; then the training set is input into the bad driving habit recognition model to train the neural network and establish classification decision rules; finally, the test set is input to test the accuracy of the bad driving habit recognition model, and the parameters of the bad driving habit recognition model are adjusted according to the test results to improve the accuracy of model recognition. When faced with numerical parameters, such as a speed value of 100km / h, the value is modified to 120km / h by adding random disturbance noise; when faced with non-numerical parameters, such as direction: left and right, since the number of the two accounts for 40% and 60% respectively in the records, the current true value of the direction is not output, but the left and right are randomly output with a probability of 40% and 60% respectively.

[0059] Establish a driver personality analysis model based on deep learning methods.

[0060] A driver personality analysis model is established, and the driver's personality profile is automatically created based on the in-car activity records and the driver's personal information to analyze the driver's driving personality characteristics. First, a driver personality analysis model is established based on a deep learning method; characteristic values ​​related to the driver's personality are extracted from the personal information of various drivers, including: gender, age, occupation, education level, driving experience and bad driving habits, and emotional keywords are extracted from the voice data in the dashcam data and in-car activity records. These are used as characteristic values ​​to input into the driver personality analysis neural network for model training, and then the driver's personality profile is output. The personality profile includes multiple dimensions: driving experience, driving preference, driving caution, driving concentration and driving patience.

[0061] Real-time analysis of bad driving habits and personality profiles of drivers around the vehicle.

[0062] The bad driving habit recognition model and the driver personality analysis model are broadcast to the local client of each user's vehicle, and the personality portrait and bad driving habits of each driver are calculated based on the current traffic violation records, in-car activity records and personal information collected locally in real time. Then the local client feeds back the personality portrait and bad driving habits of each driver to the server. The server calculates the spatial distribution of each vehicle on the road based on the positioning of each vehicle, and then feeds back the personality portrait and bad driving habits of drivers on other vehicles that are adjacent to the current vehicle to the local client based on the spatial distribution data. For example: Driver A has 6 red light violations in his traffic violation record, of which two were fined only 200 yuan, three were fined 200 yuan and deducted 3 points, and one was fined 200 yuan and deducted 6 points; because the classification decision rule of the bad driving habit recognition model will be judged as having a bad driving habit of running a red light if the driver runs a red light more than 3 times or has a fine record, the system judges that driver A has a bad driving habit of running a red light.

[0063] Step 104: Analyze road hazard factors during driving based on an image processing algorithm.

[0064] A road condition analysis model is established based on the target recognition algorithm to identify and classify objects in the vehicle's field of view, and obtain the road condition analysis data of the current road. For example, Driver A is a 40-year-old male truck driver with ten years of driving experience. He graduated from high school and has bad driving habits such as changing lanes illegally. The speed recorded in the driving recorder is much higher than the average level. The voice keywords recorded in his car contain a large number of irritable emotional words. Through the analysis of the driver personality analysis model, the personality portrait of Driver A is obtained as follows: rich driving experience, high-speed driving preference, low driving caution, low driving concentration, and low driving patience.

[0065] Establish a road condition analysis model based on target recognition algorithm.

[0066] First, a large number of images of the road environment outside the vehicle are collected by the on-board camera as training data, which are divided into a training set and a test set; then the training set is preprocessed to reduce invalid and interfering information in the image, and the image preprocessing includes: grayscale processing and histogram equalization. Then, the preprocessed image is segmented based on the simple linear iterative clustering SLIC algorithm, and the image is segmented into multiple superpixels according to the area occupied by various objects in the image. The superpixels generated by image segmentation are manually annotated, and the object category in the candidate frame is used as a label, and the label value is various traffic condition influencing factors, and the traffic condition influencing factors include: zebra crossings, speed bumps, traffic lights, roadblocks, wide-angle lenses, guardrails, car blocks, toll booths, traffic signs, traffic prohibition signs, traffic guide signs, traffic warning signs, road construction safety signs, tourist area signs, auxiliary signs, road bumps, water and snow, and passing pedestrians and animals. Then, the labeled superpixels are input into the CNN image classifier for training. The CNN image classifier extracts the image features of the superpixels corresponding to each label and establishes classification decision rules. Finally, the test set is input into the CNN image classifier, and the classification test is performed according to the classification decision rules established by the CNN image classifier to evaluate the accuracy of the classification results and adjust the model parameters according to the test results. For example, when the above two drivers, driver A and driver A, are driving side by side on the highway, the system obtains the positions of the two cars based on the on-board GPS and finds that the two cars are driving in the same direction next to each other. Therefore, the system feeds back the personality portraits and bad driving habits data obtained from the local clients of the two drivers to the local clients of the other party.

[0067] Identify factors affecting traffic conditions around the vehicle in real time.

[0068] The local client of the vehicle downloads the road condition analysis model from the server, and then identifies and classifies objects in the vehicle's field of view in real time based on the images collected by the on-board camera, discovers the factors affecting the traffic conditions on the current road, and counts the location, number and distance of the factors affecting the traffic conditions to obtain road condition analysis data.

[0069] Step 105, calculate the probability of various types of traffic accidents based on the bad driving habit data and personality profiles of surrounding drivers and the road condition analysis data.

[0070] Establish a traffic accident case library, and establish a traffic accident analysis model based on the traffic accident case library. Obtain real-time analysis data on the bad driving habits and personality of drivers of surrounding vehicles and analysis data on the current road conditions, input them into the traffic accident analysis model, and calculate various types of possible traffic accidents.

[0071] Establish a traffic accident case database.

[0072] First, the time, location, weather, cause of the accident, road condition images, personal information of the drivers of the vehicles involved and surrounding vehicles, traffic violation records, and in-car activity records of each traffic accident are collected; then the road condition images are input into the road condition analysis model to obtain the road condition analysis data of the traffic accident location; then the personal information and in-car activity records of the drivers of the vehicles involved and surrounding vehicles are input into the driver personality analysis model, and the traffic violation records are input into the bad driving habit recognition model to obtain the personality portraits and bad driving habit data of the drivers of the vehicles involved and surrounding vehicles. Finally, the time, location, weather, accident type, road condition analysis data of each traffic accident, as well as the personality portraits and bad driving habit data of the drivers of the vehicles involved and surrounding vehicles are entered into the traffic accident case database.

[0073] Establish a traffic accident probability prediction model.

[0074] Based on the traffic accident case library and the support vector machine (SVM) classification prediction model, a traffic accident probability prediction model is constructed to analyze the probability of various types of traffic accidents. First, various types of traffic accident cases are used as samples to obtain the influencing factors of traffic accidents, including: time, location, weather, road condition analysis data, and personality portraits and bad driving habits of the drivers of the involved vehicles and surrounding vehicles; then a single-classification SVM model is constructed based on the influencing factors of traffic accidents; then the activation function is used to map the value interval of the distance from the intermediate output sample of the single-classification SVM model to the center of the hypersphere in the model to [0,1], and the mapping result is the probability of a certain type of traffic accident with a surrounding vehicle.

[0075] Calculate the probability of various traffic accidents in real time.

[0076] The vehicle loads the traffic accident probability prediction model from the server to the local computer, and collects the time, location, weather, and road condition analysis data of the current location. It then obtains the bad driving habits and personality portraits of the drivers of the surrounding vehicles from the server, inputs the traffic accident probability prediction model, and calculates in real time the probability of a certain type of traffic accident between the current vehicle and the surrounding vehicles in front, behind, left, right, and four diagonal corners. It then sorts the probabilities of various types of traffic accidents for each vehicle, generates 8 traffic accident probability tables corresponding to the front, behind, left, right, and four diagonal corners, and dynamically updates the sorting. Finally, it extracts the traffic accident probability ranked first in the 8 traffic accident probability tables, and forms a 3X3 danger matrix according to the vehicle orientation, where the element in the center of the matrix corresponds to the current vehicle, with a value of 1, and the other 8 elements correspond to vehicles in 8 orientations, and are filled with corresponding values.

[0077] Step 106 , planning a driving operation plan based on the current potential traffic accident probability.

[0078] Based on the deep learning method, a driving operation planning model is established, and several subsequent driving operation plans are planned according to the current danger matrix, and feedback is given to the driver in real time. For example: in various traffic accident cases, drivers who exceed the speed limit are prone to running red lights, rear-end collisions, turning off the road, and colliding with roadside facilities. The probability of accidents rises sharply, especially when the weather, road conditions, time and location are not good. Therefore, on a certain highway, if you encounter a speeding driver, combined with the current driver's bad driving habits and personality portrait, as well as the current time, location, weather, and road condition analysis data, the SVM classification prediction model is used to analyze the probability changes of various traffic accidents in real time, and it can basically be judged that the probability of a traffic accident is very high; the highest probability is running a red light, followed by colliding with roadside facilities and turning off the road, and the lowest probability is rear-end collision.

[0079] Build a driving maneuver planning model.

[0080] Based on the deep learning method, a driving operation planning model is established. First, the danger matrix in a large amount of user historical data is collected, and the elements of the danger matrix are extracted. The name, probability value, and element coordinates of the traffic accident in the danger matrix elements are used as features, and the standard driving operations for dealing with traffic accidents in the danger matrix elements are used as labels to train the driving operation planning model. For example: when the current vehicle is on a straight highway, the time, location, weather, and road condition analysis data of the current location are collected, and combined with the bad driving habits and personality portraits of the drivers of the surrounding vehicles, the traffic accident probability table with the vehicle in front is obtained {injury due to sudden braking: 10%, collision with green belt: 7%, rear-end collision: 4%, scratches: 3%, collision: 1%, rollover: 0.1%}; combined with the traffic accident probability table of other vehicles around, the danger matrix is ​​obtained {injury due to sudden braking: 10%, tail-swinging: 15%, rollover: 10%; collision with green belt: 7%, own vehicle: 1, rear-end collision: 4%; scratches: 3%, injury due to sudden braking: 1%, collision with pedestrians: 0.1%}.

[0081] Plan driving maneuvers.

[0082] The server side loads the driving operation planning model to each vehicle client. The vehicle client plans the driving operation plan in real time according to the current vehicle's danger matrix data, and notifies the driver to implement the driving operation plan through voice prompts. The driving operation plan is updated in real time based on the danger matrix data and the driver's driving operation changes; when the danger matrix data is updated or the system captures the driver's driving operation, the system determines that the current node is for driving operation plan planning and re-plans the next driving operation plan.

Claims

1. A driving behavior road condition warning method based on federated learning, characterized in that: The method comprises: Automatically identify the driver and collect information about the driver's driving behavior; The driver information is collected, transmitted and protected during use based on a differential privacy algorithm combined with federated learning. The driver information is collected, transmitted and protected during use based on a differential privacy algorithm combined with federated learning. Specifically, the driver information is collected, transmitted and protected during use based on a differential privacy algorithm combined with federated learning. Specifically, the neural network model is trained based on federated learning, and noise is added to the client model parameters based on the differential privacy algorithm. Analyzing the driver's driving habits and personality based on a deep learning method, wherein the analysis of the driver's driving habits and personality based on a deep learning method specifically includes: training a bad driving habit recognition model based on traffic violation records, establishing a driver personality analysis model based on a deep learning method, and analyzing the bad driving habits and personality portraits of drivers around the vehicle in real time; Analyzing road hazard factors during driving based on an image processing algorithm, wherein the analyzing road hazard factors during driving based on an image processing algorithm specifically includes: establishing a road condition analysis model based on a target recognition algorithm to identify in real time factors affecting the traffic conditions of the vehicle's surrounding environment; Calculating the probability of various types of traffic accidents based on the bad driving habit data and personality profiles of surrounding drivers and road condition analysis data, specifically including: establishing a traffic accident case library, establishing a traffic accident probability prediction model, and calculating the probability of various types of traffic accidents in real time; Planning a driving operation plan according to the current potential traffic accident probability, wherein the planning of the driving operation plan according to the current potential traffic accident probability specifically includes: establishing a driving operation planning model and planning a driving operation plan; The real-time calculation of the probability of various types of traffic accidents specifically includes: The vehicle loads the traffic accident probability prediction model from the server to the local computer, and collects the time, location, weather, and road condition analysis data of the current location. It then obtains the bad driving habits and personality portraits of the drivers of the surrounding vehicles from the server, inputs the traffic accident probability prediction model, and calculates in real time the probability of a certain type of traffic accident between the current vehicle and the surrounding vehicles in front, behind, left, right, and four diagonal corners. It then sorts the probabilities of various types of traffic accidents for each vehicle, generates 8 traffic accident probability tables corresponding to the front, behind, left, right, and four diagonal corners, and dynamically updates the sorting. Finally, it extracts the traffic accident probability ranked first in the 8 traffic accident probability tables, and forms a 3X3 danger matrix according to the vehicle orientation, where the element in the center of the matrix corresponds to the current vehicle, with a value of 1, and the other 8 elements correspond to vehicles in 8 orientations, and are filled with corresponding values.

2. The method according to claim 1, wherein: The automatic identification of the driver's identity and collection of the driver's driving behavior information includes: Use the in-car camera to capture the driver's facial image when the driver enters the cab, and determine the driver's identity through face recognition technology; after determining the identity, collect driving behavior information for the current driver, first obtain the driver's traffic violation penalty record by calling the traffic management bureau's query interface; then collect the driver's current various in-car activity records through the in-car camera and driving recorder; finally, save the collected traffic violation penalty records and in-car activity records to the current driver's account. If the current driver does not have an account, indicating that this is the first time using this system, then create a new account before recording the information.

3. The method according to claim 1, wherein: The driver information collection, transmission and privacy protection during use based on differential privacy algorithm combined with federated learning include: The driver's driving behavior information and traffic violation records collected during driving are protected by differential privacy algorithm combined with federated learning; each neural network model is trained based on the federated learning method; first, the neural network model is trained locally on the client to obtain local model parameters; then, random noise is added to the local model parameters obtained after training based on the differential privacy algorithm; the server aggregates the local model parameters received from the client to obtain new global model parameters; Finally, the server broadcasts the new global model parameters to each client. The client accepts the new global model parameters and recalculates locally. This includes: training the neural network model based on federated learning; adding noise to the client model parameters based on the differential privacy algorithm; The training of the neural network model based on federated learning specifically includes: Based on the federated learning method, each neural network model is trained, and the neural network model includes: a neural network model for analyzing the driver's driving habits and personality, a road condition analysis model, a traffic accident analysis model, and a driving operation planning model; first, the server broadcasts each initial neural network model to the local client of each vehicle; then the local client trains each initial neural network model based on the driver's personal information and driving data collected locally, and feeds back the local model parameters obtained after training to the server; then the server aggregates the local model parameters received from the local client using the FedAVG algorithm to obtain new global model parameters; finally, the server broadcasts the new global model parameters to each local client; the local client accepts the new global model parameters and updates each initial neural network model, and re-performs local calculation and the next round of model iteration; The method of adding noise to the client model parameters based on the differential privacy algorithm specifically includes: After the neural network model is trained locally on the client based on federated learning to obtain the model parameters, the differential privacy algorithm is used to add random noise to the local model parameters obtained after training; for the numerical data in the local model parameters, random dynamic disturbance noise is added to the data through the Laplace mechanism; for non-numerical, discrete data, an exponential mechanism is used to determine the probability of the output result, and the output probability is determined according to the proportion of discrete data. The higher the proportion, the higher the corresponding discrete data output probability.

4. The method according to claim 1, wherein: The driver's driving habits and personality analysis based on the deep learning method includes: An initial neural network model for analyzing the driver's driving habits and personality is established based on a deep learning method, and the initial neural network model includes: a bad driving habit recognition model and a driver personality analysis model; the above two models are then broadcast to the local client of the user's vehicle, and the driving-related personality characteristics and driving habits of each driver are calculated based on local data; the model parameters are then adjusted based on the calculation results, and the updated model is fed back to the server; finally, the server aggregates the local model received from the local client to obtain a new global model parameter, and broadcasts it to the local client again; including: training a bad driving habit recognition model based on traffic violation records; establishing a driver personality analysis model based on a deep learning method; and analyzing the bad driving habits and personality portraits of drivers around the vehicle in real time; The bad driving habit recognition model is trained based on traffic violation records, specifically including: The bad driving habit recognition model is trained based on the current driver's historical traffic violation records and deep learning methods; first, a large number of traffic violation records of various drivers are collected as training sets and test sets; then the training set is cleaned, converted and features are extracted, and the extracted features include: violation factors, penalty types, fine types, whether points are deducted, deduction values, and number of violations; then the samples are labeled and classified, and the samples are marked with various bad driving habit labels, and the bad driving habit labels include: driving in the opposite direction, running a red light, changing lanes illegally, speeding, illegal parking, crossing the line, not wearing a seat belt, drunk driving, talking on the phone while driving, smoking, and deliberately covering the license plate; then the training set is input into the bad driving habit recognition model to train the neural network and establish classification decision rules; finally, the test set is input to test the accuracy of the bad driving habit recognition model, and the parameters of the bad driving habit recognition model are adjusted according to the test results to improve the accuracy of model recognition; The driver personality analysis model is established based on the deep learning method, specifically including: A driver personality analysis model is established, and the driver's personality profile is automatically created based on the in-car activity records and the driver's personal information to analyze the driver's driving personality characteristics. First, a driver personality analysis model is established based on a deep learning method. Characteristic values ​​related to the driver's personality are extracted from the personal information of various drivers, including: gender, age, occupation, education level, driving experience and bad driving habits. Emotional keywords are extracted from the driving recorder data and the voice data in the in-car activity records. These are used as characteristic values ​​to input into the driver personality analysis neural network for model training, and then the driver personality profile is output. The personality profile includes multiple dimensions: driving experience, driving preference, driving caution, driving concentration and driving patience. The real-time analysis of the bad driving habits and personality profiles of drivers around the vehicle specifically includes: The bad driving habit recognition model and the driver personality analysis model are broadcast to the local client of each user's vehicle, and the personality portrait and bad driving habits of each driver are calculated based on the current traffic violation records, in-vehicle activity records and personal information of each driver collected locally in real time; then the local client feeds back the personality portrait and bad driving habits of each driver to the server; the server calculates the spatial distribution of each vehicle on the road based on the positioning of each vehicle, and then feeds back the personality portraits and bad driving habits of drivers in other vehicles that are adjacent to the current vehicle to the local client based on the spatial distribution data.

5. The method according to claim 1, wherein: The method of analyzing road hazard factors during driving based on an image processing algorithm includes: Establishing a road condition analysis model based on a target recognition algorithm, identifying and classifying objects in the vehicle's field of view, and obtaining road condition analysis data for the current road; including: establishing a road condition analysis model based on a target recognition algorithm; identifying factors affecting the traffic conditions in the vehicle's surroundings in real time; The road condition analysis model is established based on the target recognition algorithm, specifically including: Firstly, a large number of images of the road environment outside the vehicle are collected through the on-board camera as training data, which are divided into a training set and a test set. Then, the training set is preprocessed to reduce invalid and interfering information in the image. The image preprocessing includes grayscale processing and histogram equalization. Then, the preprocessed image is segmented based on the simple linear iterative clustering SLIC algorithm, and the image is segmented into multiple superpixels according to the area occupied by various objects in the image. The superpixels generated by the image segmentation are manually annotated, and the object category in the candidate frame is used as a label, and the label value is various traffic condition influencing factors, and the traffic condition influencing factors include: zebra crossing, deceleration Traffic belts, traffic lights, roadblocks, wide-angle lenses, guardrails, car blocks, toll booths, traffic signs, traffic prohibition signs, traffic guide signs, traffic warning signs, road construction safety signs, tourist area signs, auxiliary signs, road bumps, water and snow, as well as passing pedestrians and animals; then, the superpixels with labels are input into the CNN image classifier for training, and the CNN image classifier extracts the image features of the superpixels corresponding to each label and establishes classification decision rules; finally, the test set is input into the CNN image classifier, and the classification test is performed according to the classification decision rules established by the CNN image classifier to evaluate the accuracy of the classification results, and adjust the model parameters according to the test results; The real-time identification of factors affecting the traffic conditions of the vehicle's surrounding environment specifically includes: The local client of the vehicle downloads the road condition analysis model from the server, and then identifies and classifies objects in the vehicle's field of view in real time based on the images collected by the on-board camera, discovers the factors affecting the traffic conditions on the current road, and counts the location, number and distance of the factors affecting the traffic conditions to obtain road condition analysis data.

6. The method according to claim 1, wherein: The calculation of various traffic accident probabilities based on the bad driving habits data and personality profiles of surrounding drivers and road condition analysis data includes: Establish a traffic accident case library, and establish a traffic accident analysis model based on the traffic accident case library; obtain the bad driving habits and personality analysis data of drivers of surrounding vehicles and the road condition analysis data of the current road in real time, input the traffic accident analysis model, and calculate various types of possible traffic accidents; including: establishing a traffic accident case library; establishing a traffic accident probability prediction model; and calculating the probability of various types of traffic accidents in real time; The establishment of a traffic accident case database specifically includes: First, the time, location, weather, cause of the accident, road condition image, personal information of the drivers of the vehicles involved and surrounding vehicles, traffic violation records and in-car activity records of each traffic accident are collected; then the road condition image is input into the road condition analysis model to obtain the road condition analysis data of the traffic accident location; then the personal information and in-car activity records of the drivers of the vehicles involved and surrounding vehicles are input into the driver personality analysis model, and the traffic violation records are input into the bad driving habit recognition model to obtain the personality portraits and bad driving habit data of the drivers of the vehicles involved and surrounding vehicles; finally, the time, location, weather, accident type, road condition analysis data of each traffic accident, as well as the personality portraits and bad driving habit data of the drivers of the vehicles involved and surrounding vehicles are entered into the traffic accident case database; The traffic accident probability prediction model is established, specifically comprising: A traffic accident probability prediction model is constructed based on the traffic accident case library and the support vector machine (SVM) classification prediction model to analyze the probability of various types of traffic accidents. First, various types of traffic accident cases are used as samples to obtain the influencing factors of traffic accidents, including: time, location, weather, road condition analysis data, and personality portraits and bad driving habits of the drivers of the involved vehicles and surrounding vehicles. Then, a single-classification SVM model is constructed based on the influencing factors of traffic accidents. Then, an activation function is used to map the value interval of the distance from the intermediate output sample of the single-classification SVM model to the center of the hypersphere in the model to [0,1]. The mapping result is the probability of a certain type of traffic accident occurring with a surrounding vehicle.

7. The method according to claim 1, wherein: The driving operation plan is planned according to the current potential traffic accident probability, including: A driving operation planning model is established based on deep learning methods, and several subsequent driving operation plans are planned according to the current danger matrix, and real-time feedback is given to the driver; including: establishing a driving operation planning model; planning a driving operation plan; The establishment of the driving operation planning model specifically includes: A driving operation planning model is established based on deep learning methods. First, the danger matrix in a large amount of user historical data is collected, and the elements of the danger matrix are extracted. The name, probability value, and element coordinates of the traffic accident in the danger matrix elements are used as features, and the standard driving operations for dealing with traffic accidents in the danger matrix elements are used as labels to train the driving operation planning model. The planned driving operation scheme specifically includes: The server side loads the driving operation planning model to each vehicle-mounted client. The vehicle-mounted client plans the driving operation plan in real time according to the danger matrix data of the current vehicle, and notifies the driver to implement the driving operation plan through voice prompts. The driving operation plan is updated in real time according to the danger matrix data and the driver's driving operation changes. When the danger matrix data is updated or the system captures the driver's driving operation, the system determines that the current node is for driving operation plan planning and re-plans the next driving operation plan.

Images