Driver dangerous driving behavior automatic alarm method and system

Abstract

The invention discloses an automatic alarm method and system for dangerous driving behaviors of a driver, and the method comprises the steps: collecting real-time video data of the driver through a vehicle-mounted camera, obtaining an undetermined video frame through preprocessing, and inputting the undetermined video frame into a dangerous driving behavior recognition model to obtain a recognition result. And if the result shows that the dangerous driving behavior exists, the vehicle-mounted terminal alarm gives an alarm and uploads the related target video frame to the preset alarm server, so that automatic monitoring and early warning of the dangerous driving behavior are realized, and improvement of road traffic safety is facilitated.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and system for automatically warning a driver of dangerous driving behavior. Background Art

[0002] As the number of cars continues to increase, road traffic safety has become an increasingly important issue. Dangerous driving behavior is one of the important factors leading to traffic accidents, such as fatigue driving and distracted driving. Traditional supervision methods mostly rely on on-site law enforcement by traffic police, which makes it difficult to achieve comprehensive and timely monitoring. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for automatically warning a driver of dangerous driving behavior.

[0004] In a first aspect, an embodiment of the present invention provides a method for automatically warning a driver of dangerous driving behavior, comprising:

[0005] The real-time video data of the driver is collected in real time through the camera configured on the vehicle terminal;

[0006] Preprocessing the real-time video data to obtain a plurality of pending video frames;

[0007] Inputting the plurality of pending video frames into a preset dangerous driving behavior recognition model respectively to obtain a recognition result corresponding to each of the pending video frames;

[0008] On the basis that the recognition result indicates the existence of dangerous driving behavior, an alarm is issued through the alarm of the vehicle-mounted terminal, and a target video frame corresponding to the dangerous driving behavior is uploaded to a preset alarm server.

[0009] In a possible implementation, the dangerous driving behavior recognition model is obtained by:

[0010] Obtaining a basic model for performing video frame recognition; wherein the basic model includes a feature extraction component and a first model network architecture;

[0011] Performing dimensionality reduction processing on the feature extraction component to obtain a dimensionality reduction model;

[0012] The dimension reduction model is transmitted to a preset training environment, so that the preset training environment forms a target recognition model according to the dimension reduction model and the second model network architecture, and the target recognition model is trained by performing a training process to optimize the second model network architecture through a video frame instance corresponding to a target dangerous driving recognition task;

[0013] Obtaining the second model network architecture of the target recognition model when the preset model completion degree is met;

[0014] According to the second model network architecture, the first model network architecture in the basic model is optimized to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0015] In a possible implementation, the first model network architecture includes a first recognition unit, and the second model network architecture includes a second recognition unit;

[0016] The method of optimizing the first model network architecture in the basic model according to the second model network architecture to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task includes:

[0017] According to the second recognition unit, the first recognition unit in the basic model is optimized to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0018] In a possible implementation, the first recognition unit in the basic model is optimized according to the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task, including:

[0019] Replacing the first recognition unit in the basic model with the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task;

[0020] or,

[0021] According to the network parameters of the second recognition unit, the network parameters of the first recognition unit in the basic model are updated to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0022] In a possible implementation, the first model network architecture further includes a first feature matrix generated according to first prior knowledge, where the first prior knowledge is prior knowledge associated with the task based on which the basic model is pre-trained;

[0023] The second model network architecture further includes a second feature matrix generated according to second prior knowledge, where the second prior knowledge is prior knowledge associated with the target dangerous driving identification task;

[0024] Before obtaining the dangerous driving behavior identification model for performing the target dangerous driving identification task, the method further includes:

[0025] According to the second feature matrix in the second model network architecture, the first feature matrix in the basic model is optimized.

[0026] In a possible implementation, optimizing the first feature matrix in the basic model according to the second feature matrix in the second model network architecture includes:

[0027] Fusing the first feature matrix in the base model with the second feature matrix in the second model network architecture to obtain a fused feature matrix;

[0028] According to the fused feature matrix, the first feature matrix in the basic model is updated to make the basic model adapt to the target dangerous driving identification task.

[0029] In a possible implementation, performing dimensionality reduction processing on the feature extraction component to obtain a dimensionality reduction model includes:

[0030] Performing model sparsification on the feature extraction component to obtain a sparse model;

[0031] The sparse model is subjected to transfer learning through the feature extraction component to obtain a dimensionality reduction model.

[0032] In a possible implementation, the obtaining of the second model network architecture of the target recognition model when the preset model completion degree is satisfied includes:

[0033] Obtaining a second model network architecture determined by the preset training environment;

[0034] Among them, the second model network architecture determined by the preset training environment is: the second model network architecture when the target recognition model meets the preset model completion degree.

[0035] In a possible implementation, the obtaining of the second model network architecture of the target recognition model when the preset model completion degree is satisfied includes:

[0036] Sending a completion status acquisition indication regarding the model completion status to the preset training environment at a preset time interval;

[0037] Obtaining feedback information sent by the preset training environment for obtaining an indication of the completion status; wherein the feedback information is obtained by the preset training environment according to the completion status of the target recognition model;

[0038] Based on the acquired feedback information indicating that the target recognition model meets the preset model completion degree, a second model network architecture of the target recognition model is extracted from the preset training environment.

[0039] In a second aspect, an embodiment of the present invention provides a server system, including a server, wherein the server is used to execute the method described in the first aspect.

[0040] Compared with the prior art, the beneficial effects provided by the present invention include: using the method and system for automatically alarming a driver's dangerous driving behavior disclosed in the present invention, the real-time video data of the driver is collected through the vehicle camera, and the pending video frames are obtained through preprocessing, and the recognition results are input into the dangerous driving behavior recognition model. If the result shows that there is a dangerous driving behavior, the vehicle terminal alarm alarms and uploads the relevant target video frames to the preset alarm server, realizing automatic monitoring and early warning of dangerous driving behavior, which helps to improve road traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0042] Figure 1 A schematic flow chart of the steps of a method for automatically alerting a driver of dangerous driving behavior provided by an embodiment of the present invention;

[0043] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0045] The specific implementation modes of the present invention are described in detail below in conjunction with the accompanying drawings.

[0046] In order to solve the technical problems in the aforementioned background technology, Figure 1 This is a flow chart of a method for automatically alerting a driver of dangerous driving behavior provided in an embodiment of the present disclosure. The method for automatically alerting a driver of dangerous driving behavior is introduced in detail below.

[0047] Step S201, collecting real-time video data of the driver through a camera configured on the vehicle terminal;

[0048] Step S202, preprocessing the real-time video data to obtain a plurality of pending video frames;

[0049] Step S203, inputting the plurality of pending video frames into a preset dangerous driving behavior recognition model respectively to obtain a recognition result corresponding to each of the pending video frames;

[0050] Step S204, based on the recognition result indicating the existence of dangerous driving behavior, an alarm is issued through the alarm of the vehicle-mounted terminal, and a target video frame corresponding to the dangerous driving behavior is uploaded to a preset alarm server.

[0051] In an embodiment of the present invention, for example, it is assumed that the server is connected to the vehicle terminals of multiple vehicles. In a car traveling on a highway, the camera of the vehicle terminal is installed in a suitable position in front of the driver's seat, and the driver's face, body movements, etc. can be clearly captured. The camera continues to shoot at a speed of 30 frames per second, and these video data are transmitted to the server in real time through the vehicle's network equipment (such as 4G / 5G module). For example, when the driver Xiao Zhang is driving, the camera continues to capture his every move, from his normal driving movements to all movements before possible dangerous driving behaviors are fully recorded.

[0052] After the server receives the video stream data from the vehicle terminal camera, it starts preprocessing. First, due to network transmission and other reasons, there may be some noise in the video data. The server uses an image filtering algorithm, such as a median filtering algorithm, to remove the salt and pepper noise in the video frame. Then, considering the high video frame rate, in order to improve the efficiency of subsequent processing, the server extracts frames according to certain rules, such as extracting one frame every 5 frames, so that multiple pending video frames are obtained from the continuous video stream. Taking Xiao Zhang's previous example, the server preprocesses the continuous video data and selects some video frames that can represent different moments of the driving process as the pending video frames for subsequent identification.

[0053] The server has a pre-set dangerous driving behavior recognition model. This model is trained with a large amount of data and can identify dangerous driving behaviors such as fatigue driving (yawning, drooping eyelids), distracted driving (looking at mobile phones, frequently turning heads to talk to passengers, etc.). The server inputs the pending video frames obtained from the previous steps into this recognition model one by one. For example, for a pending video frame of Xiao Zhang, the model will analyze his facial expression. If his eyes are half-closed and last for a period of time, the model will preliminarily determine that this video frame may have the characteristics of fatigue driving. Then all pending video frames are subjected to such recognition processing to obtain the recognition results corresponding to each pending video frame.

[0054] If a video frame is found to contain dangerous driving behavior in the previous recognition results, for example, Xiao Zhang is identified as looking down at his phone in a video frame, the server will send a command to the vehicle terminal. After the vehicle terminal receives the command, the alarm sounds to remind Xiao Zhang that this is a dangerous driving behavior. At the same time, the server will upload the target video frame that is determined to contain dangerous driving behavior (Xiao Zhang looking at his phone) to the preset alarm server. The preset alarm server can be a server specially used by the traffic management department to collect evidence of dangerous driving for subsequent investigations, statistics, and other work.

[0055] In an embodiment of the present invention, the dangerous driving behavior recognition model is obtained in the following manner and can be implemented through the following examples.

[0056] Obtaining a basic model for performing video frame recognition; wherein the basic model includes a feature extraction component and a first model network architecture;

[0057] Performing dimensionality reduction processing on the feature extraction component to obtain a dimensionality reduction model;

[0058] The dimension reduction model is transmitted to a preset training environment, so that the preset training environment forms a target recognition model according to the dimension reduction model and the second model network architecture, and the target recognition model is trained by performing a training process to optimize the second model network architecture through a video frame instance corresponding to a target dangerous driving recognition task;

[0059] Obtaining the second model network architecture of the target recognition model when the preset model completion degree is met;

[0060] According to the second model network architecture, the first model network architecture in the basic model is optimized to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0061] In an embodiment of the present invention, illustratively, in the early stage of building a dangerous driving behavior recognition model, the server must first obtain a basic model for performing video frame recognition. This basic model can be selected from an existing model library. For example, the server selects a convolutional neural network (CNN) model as a basic model from an open source deep learning model library. This CNN model includes a feature extraction component, which is used to extract various features from the input video frames, such as texture features, shape features, etc. The first model network architecture specifies the overall structure of this model, including the settings of convolutional layers, pooling layers, fully connected layers, etc. and the connection methods between them.

[0062] Taking the identification of driver fatigue driving as an example, the feature extraction component of the basic model will focus on the driver's eye area, extracting the eye contour, pupil features, etc. The convolution layer in the first model network architecture will perform convolution operations on the input video frames to detect eye feature patterns of different sizes and directions. The pooling layer will downsample these features to reduce the amount of data while retaining key features. The fully connected layer is used to make a comprehensive judgment on these features and preliminarily determine whether there are signs of fatigue driving.

[0063] The server performs model sparsification on the feature extraction component in the base model. In this process, the server uses a sparsification algorithm, such as the L1 regularization algorithm. For the weight parameters in the feature extraction component, L1 regularization will cause some unimportant weights to approach zero. Assume that a convolution kernel in the feature extraction component contributes very little to identifying dangerous driving behavior when extracting the driver's facial features. After L1 regularization, the weight corresponding to this convolution kernel will gradually become close to zero. In this way, a sparse model is obtained in which some unimportant feature extraction paths are weakened or removed, reducing the complexity of the model.

[0064] The server uses the feature extraction component to perform transfer learning on the sparse model. Since the basic model has been pre-trained on some related image recognition tasks, it has certain knowledge. For example, it may have learned some general knowledge about facial feature recognition. During the transfer learning process, the server adjusts the parameters in the sparse model to adapt to the dangerous driving behavior recognition task. Taking the recognition of driver distracted driving as an example, the feature extraction component further adjusts the weights based on the sparse model so that it can better extract features related to distracted driving from the video frame, such as the relative position of the driver's hand and the mobile phone. After this process, a dimensionality reduction model is obtained.

[0065] The preset training environment in the server can be a computing cluster dedicated to deep learning model training, which is equipped with a powerful graphics processing unit (GPU) to accelerate the training process. The server transfers the dimensionality reduction model to this training environment. At the same time, in this training environment, the second model network architecture is also predefined. This second model network architecture is specially designed according to the characteristics of the dangerous driving behavior recognition task. For example, it may focus more on the analysis of the driver's behavior sequence and has a special recurrent neural network (RNN) structure to process the time series information of the video frame.

[0066] In the training environment, the dimensionality reduction model and the second model network architecture are combined into a target recognition model. Then, a large number of video frame instances corresponding to the target dangerous driving recognition task are used for training. These video frame instances contain various dangerous driving behaviors (such as fatigue driving, distracted driving, etc.) as well as normal driving situations. For example, there is a group of video frame instances showing the driver's facial expressions and driving movements under different degrees of fatigue, and some video frame instances show the driver's state during normal driving. During the training process of the target recognition model, the parameters of the model are continuously adjusted according to these instances. For the RNN part in the second model network architecture, if the prediction result is found to be inaccurate when processing the video frame sequence of fatigue driving, the weight of the RNN will be adjusted to better capture the change law of fatigue driving behavior over time, thereby optimizing the second model network architecture.

[0067] The preset training environment will continuously evaluate the completion of the target recognition model during the training process. This completion can be measured by a variety of indicators, such as accuracy, recall, etc. When the target recognition model meets the preset model completion requirements, for example, the accuracy reaches more than 90%, the training environment will store the second model network architecture information at this time. The server directly obtains this stored second model network architecture from the training environment.

[0068] The server sends a completion status acquisition indication regarding the model completion status to the preset training environment at a certain time interval (such as every hour). After receiving the indication, the training environment will check the completion status of the target recognition model. If the model does not meet the requirements at this time, the training environment will return the current model status information to the server. When the target recognition model meets the preset model completion, the training environment will send a return information for the completion status acquisition indication to the server. This return information includes a second model network architecture that meets the requirements. Based on the obtained return information, the server extracts the second model network architecture of the target recognition model from the preset training environment.

[0069] Assume that the first model network architecture includes a first recognition unit, and the second model network architecture includes a second recognition unit. If the second recognition unit has better performance in processing dangerous driving behavior recognition tasks, for example, it has higher accuracy in recognizing distracted driving behaviors such as drivers looking at their mobile phones. The server will optimize the first recognition unit in the basic model based on the second recognition unit. One way is to directly replace the first recognition unit in the basic model with the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task. Another way is to update the network parameters of the first recognition unit in the basic model based on the network parameters of the second recognition unit. For example, if the second recognition unit has a more accurate network parameter setting for judging the relationship between the driver's hand movements and the position of the mobile phone, the server will update these parameters to the first recognition unit so that the optimized model can better perform the dangerous driving behavior recognition task.

[0070] The first model network architecture also includes a first feature matrix generated according to the first prior knowledge, and the second model network architecture also includes a second feature matrix generated according to the second prior knowledge. Before optimizing the first model network architecture, the server optimizes the first feature matrix in the basic model according to the second feature matrix in the second model network architecture. Specifically, the server fuses the first feature matrix in the basic model with the second feature matrix in the second model network architecture to obtain a fused feature matrix. For example, the first feature matrix contains some knowledge about the overall features of the face during pre-training of the basic model, and the second feature matrix contains knowledge about the relationship between the key facial areas (such as eyes, mouth, etc.) of the driver and the dangerous driving behavior in the dangerous driving behavior recognition task. The server fuses the two feature matrices, and then updates the first feature matrix in the basic model according to the fused feature matrix, so that the basic model adapts to the target dangerous driving recognition task, and finally obtains a dangerous driving behavior recognition model for performing the dangerous driving behavior recognition task.

[0071] In an embodiment of the present invention, the first model network architecture includes a first recognition unit, and the second model network architecture includes a second recognition unit; the first model network architecture in the basic model is optimized according to the second model network architecture to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task, which can be implemented through the following examples.

[0072] According to the second recognition unit, the first recognition unit in the basic model is optimized to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0073] In an embodiment of the present invention, illustratively, in the process of constructing a dangerous driving behavior recognition model, the recognition unit plays a vital role. For the first recognition unit in the first model network architecture, it is a structure pre-set based on the basic model, and is intended to perform preliminary feature analysis on the input video frame and preliminary judgment of dangerous driving behavior. For example, when the basic model is initially used for general video frame recognition tasks, the first recognition unit will broadly classify the various elements in the video frame to determine whether there are objects such as faces and hands that may be related to driving behavior.

[0074] The second recognition unit in the second model network architecture is specially designed and optimized for dangerous driving recognition tasks. In dangerous driving scenarios, it focuses more on the specific behavioral characteristics of the driver. For example, for the recognition of fatigue driving, the second recognition unit will focus on analyzing key features such as the driver's eye state (such as the degree of eyelid opening and closing, blinking frequency, etc.), the amplitude of head shaking, etc.; for distracted driving, it will accurately detect the contact between the driver's hands and mobile phones or other non-driving related items, the angle of sight deviation from the driving direction, etc.

[0075] In some cases, the server will directly replace the first recognition unit in the basic model with the second recognition unit to obtain a model suitable for the task of identifying dangerous driving behaviors. For example, when processing driver fatigue driving identification, the first recognition unit of the basic model may originally focus on the recognition of the entire cockpit scene, including the recognition of various elements such as the dashboard and seats. This type of recognition is not very targeted for the identification of dangerous driving behaviors. The second recognition unit is specially constructed for driver fatigue characteristics, and it can directly focus on the driver's facial features, especially the eye area. After the server replaces the first recognition unit with the second recognition unit, the model can directly perform accurate analysis of the driver's eye state when receiving the video frame input, and no longer waste computing resources on the recognition of irrelevant cockpit elements.

[0076] Taking distracted driving as an example, the first recognition unit may have focused on the outlines of large objects in the video frame in the previous basic tasks, and the recognition of small objects such as mobile phones in the driver's hands was not accurate enough. The second recognition unit is specially designed to accurately identify whether the driver is holding a mobile phone, whether the mobile phone screen is lit, and other features closely related to distracted driving. After the server is replaced, the new model can quickly and accurately determine whether there is distracted driving behavior when facing video frame input, greatly improving the efficiency and accuracy of identifying dangerous driving behaviors.

[0077] The network parameters determine the recognition unit's ability to extract and analyze the features of the input video frame. During the training process of the second recognition unit for the dangerous driving behavior recognition task, its network parameters are adjusted to the state that is most suitable for recognizing the relevant features of dangerous driving behavior. For example, when recognizing fatigue driving, the weight parameters of the convolutional layer related to eye feature extraction in the second recognition unit, after training with a large number of fatigue driving video frame examples, have been able to accurately highlight the key features of the eye, such as the changing trend of the pupil, the change in the ratio of the white of the eye to the eyeball, etc.

[0078] The server updates the network parameters of the first recognition unit based on these network parameters of the second recognition unit. For the corresponding convolutional layer in the first recognition unit, the server will copy or adjust the weight parameters of the convolutional layer in the second recognition unit according to certain rules and apply them to the first recognition unit. Taking distracted driving recognition as an example, assuming that the second recognition unit has a set of accurate fully connected layer network parameters for judging the direction of the driver's line of sight, the server will update these parameters to the fully connected layer of the first recognition unit. In this way, the first recognition unit can learn the advantages of the second recognition unit in identifying dangerous driving behaviors without changing its overall structure, thereby improving its ability to recognize dangerous driving behaviors.

[0079] After updating the network parameters, the performance of the first recognition unit in the dangerous driving behavior recognition task has been significantly improved. For example, in the recognition of distracted driving behaviors such as drivers looking at their phones, before updating the parameters, the first recognition unit may only vaguely determine that the driver has an object in his hand, but cannot determine whether it is a phone and whether he is operating the phone. After updating the parameters, it can accurately determine that the driver is holding a phone and looking at the phone screen, just like the second recognition unit, thereby accurately identifying distracted driving behaviors, making the entire dangerous driving behavior recognition model run more efficiently and accurately.

[0080] In an embodiment of the present invention, the first recognition unit in the basic model is optimized according to the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task, which can be implemented through the following examples.

[0081] Replacing the first recognition unit in the basic model with the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task;

[0082] or,

[0083] According to the network parameters of the second recognition unit, the network parameters of the first recognition unit in the basic model are updated to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0084] In an embodiment of the present invention, exemplarily, when constructing a dangerous driving behavior recognition model, the server first has a basic model. This basic model may be a general image recognition model, for example, it was originally built to recognize people and objects in various scenes. The first recognition unit in the basic model is also designed based on this general purpose. When processing video frames, it will operate according to the general image feature extraction and classification method. For example, it will divide the input video frame into multiple areas, extract features for each area, and then determine whether there are certain common objects, such as faces, vehicle parts, etc., based on these features. However, for the specific target task of dangerous driving behavior recognition, this general recognition method may not be accurate enough. Dangerous driving behavior recognition needs to focus more on the specific behavior of the driver, such as facial expressions during fatigue driving, hand movements during distracted driving, etc.

[0085] The second recognition unit is built specifically for the task of identifying dangerous driving behaviors. It is specially designed to be highly sensitive to features related to dangerous driving behaviors. For example, when identifying fatigue driving, the structure and parameters of the second recognition unit are adjusted to focus on the driver's eye features. It can contain submodules specifically for detecting eyelid shape, pupil size changes, and blinking frequency. When identifying distracted driving, it will focus on analyzing the driver's hand position, hand movements, and line of sight.

[0086] The server decides to replace the first recognition unit in the basic model with the second recognition unit. When this replacement operation is performed, the processing flow of the entire model is fundamentally changed. Taking an actual scenario as an example, assuming that when the basic model processes the video frames from the vehicle-mounted camera, the first recognition unit will first identify the environment of the entire cockpit, including the color of the seat, the position of the pointer on the dashboard and other information, which is not critical for the identification of dangerous driving behaviors. After replacing it with the second recognition unit, the model will immediately focus on the driver after receiving the video frame. For example, when the driver begins to show signs of fatigue and his eyelids gradually droop, the second recognition unit can quickly capture the changes in this eye feature because its structure and algorithm are specially designed for this type of dangerous driving behavior-related features. This replacement enables the model to perform dangerous driving behavior identification tasks more directly and effectively, improving the accuracy and timeliness of identification.

[0087] In the neural network model, the network parameters determine how the model processes the input data and the results. The network parameters of the second recognition unit are obtained through training with a large number of video frames related to dangerous driving behaviors. These parameters contain important information about how to accurately extract the characteristics of dangerous driving behaviors from video frames. For example, when identifying fatigue driving, the weight parameters of the convolutional layer related to eye feature extraction in the second recognition unit are optimized by learning from many video frames of the driver's eyes in fatigue driving conditions. These weight parameters can highlight the fatigue-related parts of the eye features, such as the thickness of the eyelids, changes in the area of ​​the whites of the eyes, etc.

[0088] After the server obtains the network parameters of the second recognition unit, it starts to update the network parameters of the first recognition unit in the basic model. Taking distracted driving recognition as an example, the first recognition unit may not have accurately recognized the driver's hand movements when processing video frames. After the second recognition unit is trained, there is a set of network parameters in its fully connected layer that can accurately determine whether there is distracted driving behavior based on the hand position, posture, and relative relationship with other objects (such as mobile phones). The server will apply this set of network parameters to the corresponding layer of the first recognition unit. This update process may need to be carried out according to certain rules. For example, according to the similarity of the structure of the first recognition unit and the second recognition unit, the network parameters are appropriately scaled or adjusted.

[0089] After updating the network parameters, the performance of the first recognition unit in the dangerous driving behavior recognition task has been significantly improved. Previously, the first recognition unit could only roughly determine that the driver's hand had movement, but could not accurately determine whether this movement was related to distracted driving. After updating the parameters, it can accurately analyze the intention of the driver's hand movements like the second recognition unit. For example, it can determine whether the driver is adjusting the radio or operating a mobile phone. The advantage of this update method is that it does not require a complete change in the structure of the first recognition unit. Instead, it absorbs the advantages of the second recognition unit in dangerous driving behavior recognition while retaining some characteristics of the basic model, so that the model can adapt to the dangerous driving behavior recognition task more quickly, while reducing the consumption of computing resources and development time during model construction.

[0090] In an embodiment of the present invention, the first model network architecture also includes a first feature matrix generated based on first prior knowledge, and the first prior knowledge is prior knowledge associated with the task based on which the basic model is pre-trained; the second model network architecture also includes a second feature matrix generated based on second prior knowledge, and the second prior knowledge is prior knowledge associated with the target dangerous driving identification task; before obtaining the dangerous driving behavior identification model for performing the target dangerous driving identification task, the following implementation method is also provided.

[0091] According to the second feature matrix in the second model network architecture, the first feature matrix in the basic model is optimized.

[0092] In the embodiment of the present invention, illustratively, the basic model in the pre-training stage may be for some general image recognition tasks, such as identifying different types of vehicles, pedestrian postures, etc. In this pre-training process, the model learns a lot of prior knowledge related to these tasks.

[0093] The first feature matrix in the first model network architecture is generated based on the first prior knowledge during pre-training. For example, the pre-training task involves identifying cars at different angles, and the first prior knowledge includes the various shape features and color distribution patterns of the car. The first feature matrix will encode this knowledge in a specific way, such as representing the contour features of the car in the matrix in the form of a specific vector. For pedestrian posture recognition, information such as the relative position relationship of key parts of the human body will be encoded into the first feature matrix.

[0094] In the basic model, the first feature matrix plays an important role in the feature extraction and preliminary judgment process of the video frame. When the basic model processes a video frame containing a driver, the first feature matrix guides the model to analyze the video frame according to the knowledge learned during pre-training. For example, it prompts the model to first look for elements in the video frame that resemble car parts or pedestrian shapes. This analysis method may be indirectly related to the dangerous driving behavior recognition task, but it is the initial analysis logic of the basic model.

[0095] The target dangerous driving recognition task focuses on specific dangerous behaviors of drivers, such as fatigue driving, distracted driving, etc. It is very different from the pre-training task of the basic model.

[0096] For fatigue driving, the second prior knowledge includes the changing patterns of various facial features of the driver when he is fatigued, such as the specific shape of the eyes (the degree of eyelid relaxation, changes in pupils, etc.), the shaking pattern of the head, etc. For distracted driving, the second prior knowledge covers the contact characteristics of the driver's hands with non-driving related objects (such as mobile phones), the angle range of the line of sight deviating from the driving direction, etc.

[0097] The second feature matrix in the second model network architecture is generated based on the second prior knowledge related to the dangerous driving identification task. This matrix encodes this knowledge in a way that is suitable for dangerous driving behavior identification. For example, in the part related to fatigue driving, it may encode the changes in eye features corresponding to different fatigue levels into the matrix in the form of a vector so that the model can quickly identify whether the driver is in a fatigued state.

[0098] The server optimizes the first feature matrix in the basic model according to the second feature matrix in the second model network architecture. A fusion operation is performed first. For example, the elements related to the driver's body parts in the first feature matrix (although these elements are generated based on the basic model pre-training task, they may have a certain correlation with dangerous driving behaviors, such as the recognition knowledge of the driver's overall profile in the basic model pre-training) are fused with the driver's body features specifically for dangerous driving behaviors in the second feature matrix (such as eye features during fatigue driving).

[0099] Assume that part of the first feature matrix represents the general shape of the driver's head, while the second feature matrix contains features about the head shaking trajectory during fatigue driving. The server will fuse these two parts of information through a specific algorithm, which may be to add the corresponding vectors of the two according to a certain weight, or use a more complex feature splicing and conversion algorithm to obtain a fused feature matrix.

[0100] After obtaining the fused feature matrix, the server updates the first feature matrix based on the fused feature matrix. For example, if certain elements in the fused feature matrix indicate that the driver's eye features are more critical to the overall judgment in the identification of dangerous driving behavior, the server will adjust the eye-related part of the first feature matrix.

[0101] Assuming that in the basic model pre-training, the first feature matrix does not encode the eye features accurately enough, and the fused feature matrix provides a more accurate way to encode the eye features, the server will replace the eye feature part in the first feature matrix with the corresponding part in the fused feature matrix, or fine-tune the first feature matrix according to the eye feature information in the fused feature matrix. In this way, the first feature matrix can better adapt to the dangerous driving behavior recognition task.

[0102] After such optimization, the basic model will be more accurate when processing video frames for dangerous driving behavior identification. Previously, due to the limitations of the first feature matrix, the basic model would ignore some subtle changes in eye features when identifying fatigue driving. The optimized first feature matrix, because it incorporates key information from the second feature matrix, can more keenly capture these features related to dangerous driving behaviors. For example, when the driver is slightly fatigued and his eyelids begin to droop slowly, the model can more promptly and accurately determine that this is a manifestation of fatigue driving, thereby improving the performance of the entire dangerous driving behavior recognition model.

[0103] In the embodiment of the present invention, the optimization of the first feature matrix in the basic model according to the second feature matrix in the second model network architecture can be implemented through the following examples.

[0104] Fusing the first feature matrix in the base model with the second feature matrix in the second model network architecture to obtain a fused feature matrix;

[0105] According to the fused feature matrix, the first feature matrix in the basic model is updated to make the basic model adapt to the target dangerous driving identification task.

[0106] In an embodiment of the present invention, exemplarily, the first feature matrix in the basic model is generated based on the prior knowledge associated with the basic model pre-training task. Assuming that the basic model pre-training is for a general scene recognition task, the first feature matrix contains feature representations of various scene elements. For example, in the part related to the driving scene, it may have certain feature encodings for the shape, color distribution, and so on of common objects in the cockpit. If the first feature matrix is ​​regarded as a multidimensional array, some dimensions may represent the approximate shape features of the driver's seat, and other dimensions represent the color features of the dashboard, etc. These features are abstracted from a large amount of image data pre-trained by the basic model. Although they have a certain indirect relationship with the identification of dangerous driving behaviors, they are not accurate enough.

[0107] The second feature matrix in the second model network architecture is generated based on the prior knowledge of the dangerous driving recognition task. For the recognition of fatigue driving, the second feature matrix has very detailed encoding in the dimensions related to the driver's face. For example, it may use specific numerical values ​​to represent the precise shape changes of the eyes under different levels of fatigue, the numerical range corresponding to the degree of eyelid relaxation, etc. For distracted driving, it has accurate feature representation in the dimensions related to the driver's hands and line of sight, such as the spatial position relationship when the hand touches the mobile phone.

[0108] The server starts the fusion operation. A simple way to fuse is to add by dimension. For example, in the dimension related to the driver's eye features, the first feature matrix may have a broader representation (because the basic model pre-training task is not specifically for fatigue driving eye features), while the second feature matrix has a very precise representation. The server adds the values ​​of the two matrices in the eye feature dimension, but this addition is not a simple arithmetic addition, but is added according to a certain weight. This weight is determined according to the importance of the dangerous driving behavior recognition task. Assuming that in this task, the eye feature in the second feature matrix has a higher weight because it is specially designed for fatigue driving, then when adding, the value of the eye feature dimension of the second feature matrix will be multiplied by a larger weight and then added to the corresponding dimension of the first feature matrix.

[0109] In addition to adding by dimension, a fusion method of feature splicing may also be used. For the feature representation of objects in the cockpit, the first feature matrix has its own unique encoding method, while the second feature matrix may not have this part of the content or the encoding is not comprehensive enough. The server can directly splice the feature part of the cockpit objects in the first feature matrix to the appropriate position of the second feature matrix to form a new fused feature matrix. In this way, the fused feature matrix contains both some common features about the cockpit environment in the basic model and the precise information about dangerous driving behaviors (such as the driver's own behavior characteristics) in the second feature matrix.

[0110] During the entire fusion process, the server needs to consider the physical meaning of each element in the feature matrix and its importance in the dangerous driving behavior recognition task. For example, for the feature of the driver's line of sight in distracted driving, this is a very critical factor. During fusion, it is necessary to ensure that the line of sight feature in the second feature matrix can be accurately integrated into the fused feature matrix and will not be disturbed by the less relevant features in the first feature matrix.

[0111] The fusion feature matrix combines the common features of the basic model and the specific features of the dangerous driving identification task. Its structure is more complex, but it is more suitable for the dangerous driving behavior identification task. For example, it can not only use some of the knowledge of the overall layout of the cockpit in the basic model (which helps to provide environmental background information when identifying dangerous driving behaviors), but also accurately capture the characteristics of the driver's dangerous driving behavior, such as subtle facial changes during fatigue driving and hand movements and line of sight deviation during distracted driving.

[0112] The first feature matrix in the basic model was originally used for the pre-training task of the basic model. Although it contains some driving-related information, it is not accurate and comprehensive enough for the dangerous driving behavior recognition task. By generating a fusion feature matrix, a feature representation that is more suitable for the dangerous driving behavior recognition task is obtained. In order for the basic model to better perform the dangerous driving behavior recognition task, the first feature matrix needs to be updated according to the fusion feature matrix.

[0113] The server updates the first feature matrix according to the information in the fused feature matrix. For example, in the part related to the driver's eye features, if the value range representing the eye fatigue feature in the fused feature matrix is ​​significantly different from the original value range in the first feature matrix, the server will update the eye feature value range in the first feature matrix to the value range in the fused feature matrix. This involves adjusting the values ​​of multiple related dimensions in the first feature matrix.

[0114] For object features in the cockpit that are indirectly related to dangerous driving behavior, such as the features of the driver's seat, if the fusion feature matrix has a new representation of the driver's seat features through concatenation or weighted addition (this new representation may take into account the influence of the driver's seat position and driver's posture on the judgment of dangerous driving behavior), the server will use this new representation to update the driver's seat feature part in the first feature matrix.

[0115] During the update process, the server must ensure the structural integrity and logical consistency of the first feature matrix. For example, the relationship between the first feature matrix and other elements cannot be destroyed by updating a feature element. If there is a certain logical association between the driver's seat feature and the driver's hand movement feature in the first feature matrix (for example, the position of the driver's seat may affect the judgment of the driver's distraction behavior such as operating a mobile phone), when updating the driver's seat feature, the maintenance of this logical association must be considered at the same time.

[0116] After the updated first feature matrix, the basic model can better adapt to the target dangerous driving identification task. Previously, the basic model may misjudge the fatigue state when identifying fatigue driving due to the inaccurate representation of eye features in the first feature matrix. Now, the updated first feature matrix can more accurately reflect the relationship between eye features and fatigue driving, thereby improving the basic model's ability to recognize fatigue driving behavior. Similarly, for distracted driving behavior, the basic model can more accurately judge whether the driver has distracted driving behavior based on features such as hand movements and line of sight in the updated first feature matrix, which improves the performance of the dangerous driving behavior recognition model as a whole.

[0117] In the embodiment of the present invention, the feature extraction component is subjected to dimensionality reduction processing to obtain a dimensionality reduction model, which can be implemented through the following examples.

[0118] Performing model sparsification on the feature extraction component to obtain a sparse model;

[0119] The sparse model is subjected to transfer learning through the feature extraction component to obtain a dimensionality reduction model.

[0120] In an embodiment of the present invention, exemplarily, the basic model in the server includes a feature extraction component, which is initially designed to widely extract various features from the input video frames. For example, when processing driving scene video frames transmitted from a vehicle-mounted camera, the feature extraction component may include multiple convolutional layers and pooling layers. There are a large number of convolutional kernels in these convolutional layers, and each convolutional kernel is trying to capture different features in the video frame, such as texture, shape, color and other features. Initially, the weight parameters of these convolutional kernels are determined based on the basic model pre-training task, and there may be some redundancy for the dangerous driving behavior recognition task.

[0121] The purpose of model sparsification is to reduce redundant information in the feature extraction component, making the model more compact and efficient. The server uses a sparsification algorithm, such as the L1 regularization algorithm. The principle of the L1 regularization algorithm is to add a penalty term for the sum of the absolute values ​​of the weights to the loss function of the model. During the training process, this penalty term will cause the weights of the model to shrink toward zero. For the convolution kernel weights in the feature extraction component, those weights that contribute less to the dangerous driving behavior recognition task will gradually approach zero.

[0122] In actual operation, the server takes all weight parameters in the feature extraction component of the base model as input and applies the L1 regularization algorithm. As training progresses, the weights of some convolution kernels begin to decrease. For example, in the base model pre-training task, there may be some convolution kernels that are specifically used to identify background elements in driving scenes, such as trees on both sides of the road, etc. These convolution kernels are not directly helpful for identifying dangerous driving behaviors (such as driver fatigue or distraction). During the sparsification process, the weights of these convolution kernels will gradually approach zero. After a certain number of training iterations, a sparse model is obtained. In this sparse model, the weights of some convolution kernels are very small, which is equivalent to greatly weakening the role of these convolution kernels in the feature extraction process, thereby reducing the complexity of the model. For example, a convolution layer with 100 convolution kernels may only have 50 convolution kernels with large weights after sparsification, and the weights of the remaining convolution kernels are close to zero, which has little effect on feature extraction.

[0123] Although the sparse model reduces redundancy, it still needs further adjustment to adapt to the dangerous driving behavior recognition task. Since the feature extraction component of the basic model has been pre-trained on other related tasks (such as general image recognition tasks), it has learned some general image feature representation knowledge. The purpose of transfer learning is to use this existing knowledge and transfer it to the dangerous driving behavior recognition task.

[0124] The server uses the feature extraction component to perform transfer learning on the sparse model. First, the server uses the weight parameters in the sparse model as the initial value. Then, the sparse model is fine-tuned using a dataset related to dangerous driving behavior recognition. This dataset contains a large number of driving scene video frames, in which various dangerous driving behaviors (such as fatigue driving, distracted driving, etc.) and normal driving situations are marked.

[0125] During fine-tuning, the feature extraction component adjusts the weights based on the new dataset. For example, for fatigue driving recognition, the feature extraction component will pay more attention to the driver's eye area. If in the sparse model, a convolution kernel originally has a certain response to the texture features of general images, during the transfer learning process, the weight of this convolution kernel will be adjusted according to the importance of eye texture features in the recognition of dangerous driving behaviors. If the eye texture feature is important for judging fatigue driving, then the weight of this convolution kernel will be adjusted in a direction that is more conducive to the recognition of eye texture features.

[0126] At the same time, for the identification of distracted driving, the feature extraction component will adjust the weights to better identify features such as the driver's hand movements and line of sight. For example, in some pooling layers in the sparse model, the downsampling parameters will be adjusted to retain more feature information related to distracted driving. Through such a transfer learning process, the feature extraction component continuously optimizes the weight parameters of the sparse model, so that the model can better extract features related to dangerous driving behaviors from video frames, thereby obtaining a dimensionality reduction model. In this dimensionality reduction model, the structure and parameters of the feature extraction component have been optimized, reducing unnecessary feature dimensions and improving the efficiency and accuracy of identifying dangerous driving behaviors.

[0127] In an embodiment of the present invention, the acquisition of the second model network architecture of the target recognition model when the preset model completion degree is met can be implemented through the following example.

[0128] Obtaining a second model network architecture determined by the preset training environment;

[0129] Among them, the second model network architecture determined by the preset training environment is: the second model network architecture when the target recognition model meets the preset model completion degree.

[0130] In an embodiment of the present invention, illustratively, the preset training environment in the server is a complex system specifically used for deep learning model training. It is built on powerful computing hardware, for example, it can include multiple high-performance graphics processing units (GPUs). These GPUs have a large number of computing cores and high-speed video memory, and can quickly process a large number of matrix operations in the neural network training process. For example, when processing the training of a dangerous driving behavior recognition model, these GPUs can calculate the forward propagation and back propagation processes of video frame data in the neural network in parallel, greatly improving the training speed.

[0131] In addition to the GPU, the preset training environment is also equipped with large-capacity memory and high-speed storage devices. The memory is used to temporarily store intermediate data during the training process, such as the feature vectors of each batch of video frame data, the weight gradient of the model, etc. High-speed storage devices (such as solid-state drives) are used to store training data sets, pre-trained models, and various configuration files during the training process.

[0132] The preset training environment uses advanced deep learning frameworks such as TensorFlow or PyTorch. These frameworks provide a series of convenient tools and functions for building, training and optimizing neural network models. In the training of the dangerous driving behavior recognition model, the framework can easily define the structure of the model, including the various layers of the second model network architecture (such as convolutional layers, pooling layers, fully connected layers, etc.) and their parameter settings. At the same time, the framework also provides optimization algorithms (such as stochastic gradient descent and its variants) to adjust the weights of the model to minimize the loss function.

[0133] In the preset training environment, in order to determine whether the target recognition model meets the preset model completion, a series of evaluation indicators need to be set. For the dangerous driving behavior recognition task, these indicators include accuracy, recall, and F1-score. Accuracy refers to the ratio of the number of video frames in which the model correctly predicts dangerous driving behavior to the total number of video frames predicted as dangerous driving behavior, which reflects the accuracy of the model's prediction. Recall is the ratio of the number of video frames in which the model correctly predicts dangerous driving behavior to the total number of video frames in which dangerous driving behavior actually exists, reflecting the model's ability to capture dangerous driving behavior. F1-score is an indicator that comprehensively considers accuracy and recall, and can more comprehensively evaluate the performance of the model.

[0134] The preset training environment uses a large number of video frame instances corresponding to the target dangerous driving recognition task for training. These video frame instances are carefully annotated to mark whether there are dangerous driving behaviors (such as fatigue driving, distracted driving, etc.) in each video frame and the specific type of dangerous driving behavior. For example, a video frame is labeled as "fatigue driving-drooping eyelids" or "distracted driving-looking at mobile phones", etc. These annotated data are divided into multiple batches, each batch contains a certain number of video frames, so that they can be gradually input into the target recognition model during the training process.

[0135] During the training process, the target recognition model (consisting of a dimensionality reduction model and a second model network architecture) receives video frame data from a batch. The model first extracts features from the video frame through the dimensionality reduction model, and then inputs the extracted features into the second model network architecture for classification prediction. For example, for a distracted driving video frame, the dimensionality reduction model will extract the relative position features of the driver's hand and the phone, the driver's line of sight, etc. The second model network architecture determines whether the video frame is a distracted driving behavior based on these features.

[0136] As training progresses, the model weights are constantly adjusted. The preset training environment calculates the gradient of each weight according to the set optimization algorithm (such as the Adam optimization algorithm), and updates the weight according to the gradient to reduce the error (loss function) between the predicted result and the true label. This process is repeated in multiple training rounds (epochs), and all training data batches are used once in each round.

[0137] After each training round, the preset training environment will calculate the performance of the target recognition model based on the set evaluation indicators (accuracy, recall, F1-score, etc.). For example, after the first round of training, the model's accuracy is calculated to be 70%, the recall is 60%, and the F1-score is 0.65. These indicators will continue to change as the number of training rounds increases. The preset training environment will continue to monitor these indicators to determine whether the model meets the preset model completion.

[0138] Assume that the preset model completion requirements are 85% accuracy, 80% recall, and 0.82 F1-score. During the training process, the various indicators of the target recognition model gradually improve. Once these preset requirements are met or exceeded, the preset training environment will determine that the target recognition model meets the preset model completion. For example, after 10 training rounds, the model's accuracy reaches 86%, the recall rate reaches 81%, and the F1-score reaches 0.83.

[0139] When the target recognition model meets the preset model completion degree, the preset training environment will determine the second model network architecture at this time. This second model network architecture contains the structural parameters of each layer (such as the size and number of convolution kernels of the convolution layer, the number of neurons in the fully connected layer, etc.) and the connection weights between layers and other information. The server obtains this determined second model network architecture from the preset training environment, so that it can subsequently optimize the first model network architecture in the basic model according to this architecture, thereby obtaining a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

[0140] In an embodiment of the present invention, the acquisition of the second model network architecture of the target recognition model when the preset model completion degree is met can be implemented through the following example.

[0141] Sending a completion status acquisition indication regarding the model completion status to the preset training environment at a preset time interval;

[0142] Obtaining feedback information sent by the preset training environment for obtaining an indication of the completion status; wherein the feedback information is obtained by the preset training environment according to the completion status of the target recognition model;

[0143] Based on the acquired feedback information indicating that the target recognition model meets the preset model completion degree, a second model network architecture of the target recognition model is extracted from the preset training environment.

[0144] In an embodiment of the present invention, illustratively, in the process of constructing a dangerous driving behavior recognition model by the server, in order to timely understand the training progress of the target recognition model, a suitable preset time interval is set. The setting of this time interval needs to comprehensively consider multiple factors, such as the scale of training data, the complexity of the model, and the availability of computing resources. Assume that for this dangerous driving behavior recognition task, due to the large amount of training data and the relatively complex model structure, the preset time interval set by the server is to send a completion status acquisition indication every 3 hours.

[0145] This completion status acquisition indication contains some necessary identification information so that the preset training environment can accurately recognize that this is a request from the server for querying the completion of the target recognition model. The indication also contains some basic information about the model, such as the model name, version number, etc. The server sends this indication to the preset training environment through an internal communication mechanism. For example, within a large data center, the server and the preset training environment are connected through a high-speed local area network, and the server uses a specific network protocol (such as the RPC protocol) to send the completion status acquisition indication to a specific service port in the preset training environment.

[0146] After receiving the completion status acquisition indication sent by the server, the preset training environment will first parse the indication to obtain the model-related information contained therein. Then, the corresponding target recognition model is found based on the information. For example, the preset training environment can maintain a model management system, which searches for the target recognition model being trained in the system based on the model name and version number.

[0147] The preset training environment evaluates the completion of the target recognition model based on the current training status. It uses pre-set evaluation indicators such as accuracy, recall, and F1-score. It is assumed that the target recognition model has been trained for a certain number of rounds when the completion status acquisition indication is received. The preset training environment obtains relevant information from the intermediate result data saved during the training process and calculates the values ​​of these evaluation indicators. For example, it counts the number of video frames in which the model correctly predicts dangerous driving behavior and the number of prediction errors on the most recent batch of test data to calculate the accuracy.

[0148] According to the completion evaluation results of the target recognition model, the preset training environment generates feedback information. If the target recognition model has not yet reached the preset model completion, the feedback information will contain the current completion index value (such as 75% accuracy, 70% recall, etc.) and some training status information, such as the number of completed training rounds, the estimated training time required, etc. If the target recognition model has met the preset model completion, the feedback information will include information related to the second model network architecture of the target recognition model in addition to the identification that the model has met the requirements. This information can include the structural parameters of each layer in the second model network architecture (such as the size and number of convolution kernels of the convolution layer, the number of neurons in the fully connected layer, etc.) and details such as the connection weights between layers. Then, the preset training environment sends the feedback information back to the server through the same communication mechanism as the received indication.

[0149] After the server receives the feedback information sent by the preset training environment, it first parses the feedback information. If the feedback information indicates that the target recognition model meets the preset model completion, the server will further extract information about the second model network architecture. The server will parse the parameters of each network layer from the feedback information in a predetermined format. For example, for the parameters of the convolution layer, the server will accurately extract parameters such as the size of the convolution kernel (such as 3×3 or 5×5), the number of convolution kernels (such as 64 or 128), and the step size of the convolution layer.

[0150] Once the relevant parameters of the second model network architecture are parsed, the server extracts the complete second model network architecture from the preset training environment. This extraction process involves interacting with the model storage system in the preset training environment. For example, the server sends a request to the model storage service in the preset training environment, requesting to obtain the second model network architecture data corresponding to the parsed parameters. The model storage service in the preset training environment reads the relevant data blocks from the storage device (such as a hard disk or distributed file system) based on the request and sends it back to the server. After receiving this data, the server successfully obtains the second model network architecture when the target recognition model meets the preset model completion degree, so that the first model network architecture in the basic model can be optimized according to this architecture in the future, thereby obtaining a dangerous driving behavior recognition model for performing dangerous driving behavior recognition tasks.

[0151] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned automatic alarm method for dangerous driving behavior of the driver. Figure 2 As shown, Figure 2 The block diagram of the computer device 100 provided in the embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112 and a communication unit 113. To achieve data transmission or interaction, the memory 111, the processor 112 and the communication unit 113 are directly or indirectly electrically connected to each other. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0152] For illustrative purposes, the foregoing description is made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Numerous modifications and variations are possible in accordance with the above teachings. These embodiments are selected and described in order to best illustrate the principles of the present disclosure and its practical application, so that those skilled in the art can best utilize the present disclosure and utilize various embodiments with different modifications to suit the intended specific application.

Claims

1. A method for automatically warning a driver of dangerous driving behavior, characterized in that: include: The real-time video data of the driver is collected in real time through the camera configured on the vehicle terminal; Preprocessing the real-time video data to obtain a plurality of pending video frames; Inputting the plurality of pending video frames into a preset dangerous driving behavior recognition model respectively to obtain a recognition result corresponding to each of the pending video frames; On the basis that the recognition result indicates the existence of dangerous driving behavior, an alarm is issued through the alarm of the vehicle-mounted terminal, and a target video frame corresponding to the dangerous driving behavior is uploaded to a preset alarm server.

2. The method according to claim 1, characterized in that The dangerous driving behavior recognition model is obtained by the following methods, including: Obtaining a basic model for performing video frame recognition; wherein the basic model includes a feature extraction component and a first model network architecture; Performing dimensionality reduction processing on the feature extraction component to obtain a dimensionality reduction model; The dimension reduction model is transmitted to a preset training environment, so that the preset training environment forms a target recognition model according to the dimension reduction model and the second model network architecture, and the target recognition model is trained by performing a training process to optimize the second model network architecture through a video frame instance corresponding to a target dangerous driving recognition task; Obtaining the second model network architecture of the target recognition model when the preset model completion degree is met; According to the second model network architecture, the first model network architecture in the basic model is optimized to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

3. The method according to claim 2, characterized in that The first model network architecture includes a first recognition unit, and the second model network architecture includes a second recognition unit; The method of optimizing the first model network architecture in the basic model according to the second model network architecture to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task includes: According to the second recognition unit, the first recognition unit in the basic model is optimized to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

4. The method according to claim 3, characterized in that The step of optimizing the first recognition unit in the basic model according to the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task includes: Replacing the first recognition unit in the basic model with the second recognition unit to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task; or, According to the network parameters of the second recognition unit, the network parameters of the first recognition unit in the basic model are updated to obtain a dangerous driving behavior recognition model for performing the target dangerous driving recognition task.

5. The method according to claim 3, characterized in that: The first model network architecture also includes a first feature matrix generated according to first prior knowledge, where the first prior knowledge is prior knowledge associated with the task based on which the basic model is pre-trained; The second model network architecture further includes a second feature matrix generated according to second prior knowledge, where the second prior knowledge is prior knowledge associated with the target dangerous driving identification task; Before obtaining the dangerous driving behavior identification model for performing the target dangerous driving identification task, the method further includes: According to the second feature matrix in the second model network architecture, the first feature matrix in the basic model is optimized.

6. The method according to claim 5, characterized in that The step of optimizing the first feature matrix in the basic model according to the second feature matrix in the second model network architecture includes: Fusing the first feature matrix in the base model with the second feature matrix in the second model network architecture to obtain a fused feature matrix; According to the fused feature matrix, the first feature matrix in the basic model is updated to make the basic model adapt to the target dangerous driving identification task.

7. The method according to claim 2, characterized in that The step of performing dimensionality reduction processing on the feature extraction component to obtain a dimensionality reduction model includes: Performing model sparsification on the feature extraction component to obtain a sparse model; The sparse model is subjected to transfer learning through the feature extraction component to obtain a dimensionality reduction model.

8. The method according to claim 2, characterized in that: The obtaining of the second model network architecture of the target recognition model when the preset model completion degree is met includes: Obtaining a second model network architecture determined by the preset training environment; Among them, the second model network architecture determined by the preset training environment is: the second model network architecture when the target recognition model meets the preset model completion degree.

9. The method according to claim 2, characterized in that: The obtaining of the second model network architecture of the target recognition model when the preset model completion degree is met includes: Sending a completion status acquisition indication regarding the model completion status to the preset training environment at a preset time interval; Obtaining feedback information sent by the preset training environment for obtaining an indication of the completion status; wherein the feedback information is obtained by the preset training environment according to the completion status of the target recognition model; Based on the acquired feedback information indicating that the target recognition model meets the preset model completion degree, a second model network architecture of the target recognition model is extracted from the preset training environment.

10. A server system, characterized in that: The method comprises a server, wherein the server is used to execute the method described in any one of claims 1 to 9.

Images