Abnormal driving behavior monitoring method, device and electronic equipment

Abstract

The present disclosure provides a method, device, and electronic device for monitoring abnormal driving behavior. The method includes inputting real-time driving video into a motion recognition model to identify, among multiple preset abnormal driving actions, a first confidence level for each abnormal driving action performed by the driver; inputting the real-time driving video into a head posture recognition model to identify, with a second confidence level, the driver's head posture as a preset abnormal head posture; determining a preset first confidence threshold corresponding to the business type, screening the abnormal driving actions for target abnormal driving actions with a first confidence level greater than the preset first confidence threshold, and sending corresponding abnormal action prompt information to the driver; and determining whether the second confidence level is greater than the preset first confidence threshold. If so, determining that the driver has experienced a distraction event and sending a distraction prompt information to the driver. The method can improve the sensitivity and accuracy of monitoring abnormal driving behavior of drivers and effectively prevent the occurrence of false positives and false negatives.

Description

Technical Field

[0001] The present disclosure relates to the field of image analysis technology, and in particular to a method, device, and electronic equipment for monitoring abnormal driving behavior. Background Art

[0002] At present, with the continuous rise of freight business, from large-scale engineering equipment to small express mail, considering the accessibility and convenience within a certain distance, more and more people choose land transportation as a logistics transportation method, and transport goods to various places by truck. This transportation method requires drivers to drive trucks for long distances. The driver's status during driving directly affects the safety of people, vehicles and goods. Therefore, monitoring the driver's status during driving has become a daily need for various logistics companies.

[0003] Due to resource limitations of on-board equipment, complex algorithms cannot be used during the current driver's driving state, which often leads to omissions and false alarms of some abnormal driving behaviors that cannot be accurately identified. In transportation services such as hazardous chemicals, which have high requirements on the driver's driving state, if omissions and false alarms occur, the risk of safety accidents will be aggravated. Therefore, how to accurately and timely monitor the driver's driving actions at the airport and provide prompts has become an urgent problem to be solved. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide a method, device, and electronic device for monitoring abnormal driving behavior, which can improve the sensitivity and accuracy of monitoring abnormal driving behavior of drivers and effectively prevent the occurrence of false alarms, missed alarms, etc.

[0005] The present disclosure provides a method for monitoring abnormal driving behavior, the method comprising:

[0006] Get real-time driving video of the driver;

[0007] Inputting the real-time driving video into a pre-trained action recognition model, and identifying a first confidence level of each abnormal driving action by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety;

[0008] Inputting the real-time driving video into a pre-trained head posture recognition model to recognize the driver's head posture, and determining a second confidence level that the head posture is a preset abnormal head posture;

[0009] determining a business type of the current driving process and a preset first confidence threshold corresponding to the business type, screening target abnormal driving actions whose first confidence is greater than the preset first confidence threshold among the abnormal driving actions, and pushing corresponding abnormal action prompt information to the driver;

[0010] It is determined whether the second confidence level is greater than the preset first confidence level threshold. If so, it is determined that the driver has experienced an attention distraction event, and attention distraction prompt information is pushed to the driver.

[0011] In an optional embodiment, after determining the business type of the current driving process and a preset first confidence threshold corresponding to the business type, screening target abnormal driving actions having the first confidence greater than the preset first confidence threshold among the abnormal driving actions, and pushing abnormal action prompt information to the driver, the method further includes:

[0012] Determining a preset second confidence threshold corresponding to the service type, wherein the preset second confidence threshold is less than the preset first confidence threshold;

[0013] Among the abnormal driving actions, screening the abnormal driving actions to be confirmed whose first confidence level is within the interval between the preset first confidence threshold and the preset second confidence threshold;

[0014] Sending the abnormal driving action to be confirmed to a preset manual confirmation platform, and receiving a confirmation result corresponding to the abnormal driving action to be confirmed returned by the manual confirmation platform, wherein the confirmation result includes a normal action and an abnormal action;

[0015] The abnormal driving actions to be confirmed whose confirmation results are abnormal actions are screened out, and corresponding abnormal action prompt information is pushed to the driver.

[0016] In an optional embodiment, after determining whether the second confidence level is greater than the preset first confidence level threshold, and if so, determining that the driver has experienced a distraction event and pushing a distraction prompt message to the driver, the method further includes:

[0017] If the second confidence level is less than the preset first confidence level threshold, determining whether the second confidence level is greater than the preset second confidence level threshold;

[0018] If it is greater than, sending the driver's head posture to the manual confirmation platform, and receiving the attention distraction confirmation result returned by the manual confirmation platform, wherein the attention distraction confirmation result includes attention distraction and attention non-distraction;

[0019] For the head posture whose confirmation result is distraction, distraction prompt information is pushed to the driver.

[0020] In an optional implementation, after obtaining the real-time driving video of the driver, the method further includes:

[0021] decoding the real-time driving video, and shortening the real-time driving video to a preset size;

[0022] Determining a frame rate and a total number of frames corresponding to the real-time driving video;

[0023] According to the frame rate and the total number of frames, time-sequential sampling is performed on the real-time driving video to obtain a plurality of video frame images.

[0024] In an optional embodiment, before inputting the real-time driving video into a pre-trained head posture recognition model to recognize the driver's head posture and determining that the head posture is a preset abnormal head posture with a second confidence level, the method further includes:

[0025] For each of the video frame images, the video frame image is converted into a corresponding grayscale image.

[0026] In an optional implementation, the action recognition model is a 3D convolutional neural network.

[0027] The present disclosure also provides a device for monitoring abnormal driving behavior, the device comprising:

[0028] Video acquisition module, used to obtain real-time driving video of the driver;

[0029] a motion recognition module, configured to input the real-time driving video into a pre-trained motion recognition model and identify a first confidence level of each abnormal driving action by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety;

[0030] a posture recognition module, configured to input the real-time driving video into a pre-trained head posture recognition model, recognize the driver's head posture, and determine a second confidence level that the head posture is a preset abnormal head posture;

[0031] an abnormal action prompting module, configured to determine a business type of the current driving process and a preset first confidence threshold corresponding to the business type, screen target abnormal driving actions whose first confidence is greater than the preset first confidence threshold among the abnormal driving actions, and push corresponding abnormal action prompt information to the driver;

[0032] The attention distraction prompt module is used to determine whether the second confidence level is greater than the preset first confidence level threshold. If so, it is determined that the driver has an attention distraction event and push attention distraction prompt information to the driver.

[0033] In an optional embodiment, the device further includes an abnormal action verification module, wherein the abnormal action verification module is specifically configured to:

[0034] Determining a preset second confidence threshold corresponding to the service type, wherein the preset second confidence threshold is less than the preset first confidence threshold;

[0035] Among the abnormal driving actions, screening the abnormal driving actions to be confirmed whose first confidence level is within the interval between the preset first confidence threshold and the preset second confidence threshold;

[0036] Sending the abnormal driving action to be confirmed to a preset manual confirmation platform, and receiving a confirmation result corresponding to the abnormal driving action to be confirmed returned by the manual confirmation platform, wherein the confirmation result includes a normal action and an abnormal action;

[0037] The abnormal driving actions to be confirmed whose confirmation results are abnormal actions are screened out, and corresponding abnormal action prompt information is pushed to the driver.

[0038] In an optional embodiment, the device further includes an attention distraction verification module, wherein the attention distraction verification module is specifically configured to:

[0039] If the second confidence level is less than the preset first confidence level threshold, determining whether the second confidence level is greater than the preset second confidence level threshold;

[0040] If it is greater than, sending the driver's head posture to the manual confirmation platform, and receiving the attention distraction confirmation result returned by the manual confirmation platform, wherein the attention distraction confirmation result includes attention distraction and attention non-distraction;

[0041] For the head posture whose confirmation result is distraction, distraction prompt information is pushed to the driver.

[0042] In an optional implementation, the device further includes a video preprocessing module, wherein the video preprocessing module is configured to:

[0043] decoding the real-time driving video, and shortening the real-time driving video to a preset size;

[0044] Determining a frame rate and a total number of frames corresponding to the real-time driving video;

[0045] According to the frame rate and the total number of frames, time-sequential sampling is performed on the real-time driving video to obtain a plurality of video frame images.

[0046] In an optional implementation, the action recognition module is further configured to:

[0047] For each of the video frame images, the video frame image is converted into a corresponding grayscale image.

[0048] An embodiment of the present disclosure also provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the above-mentioned abnormal driving behavior monitoring method or the steps in any possible implementation of the above-mentioned abnormal driving behavior monitoring method are performed.

[0049] An embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program executes the above-mentioned abnormal driving behavior monitoring method, or the steps of any possible implementation of the above-mentioned abnormal driving behavior monitoring method.

[0050] The disclosed embodiments provide a method, device, and electronic device for monitoring abnormal driving behavior. These methods include acquiring a real-time driving video of a driver; inputting the real-time driving video into a pre-trained action recognition model to identify, among multiple preset abnormal driving actions, a first confidence level for each abnormal driving action, wherein an abnormal driving action is an action that affects driving safety; inputting the real-time driving video into a pre-trained head posture recognition model to identify the driver's head posture and determine, with a second confidence level, that the head posture is a preset abnormal head posture; determining the business type of the current driving process and a preset first confidence threshold corresponding to the business type; screening, among the abnormal driving actions, target abnormal driving actions with a first confidence level greater than the preset first confidence threshold, and sending corresponding abnormal action prompt information to the driver; and determining whether the second confidence level is greater than the preset first confidence threshold. If so, determining that the driver has experienced a distraction event and sending a distraction prompt information to the driver. These methods can improve the sensitivity and accuracy of monitoring abnormal driving behavior, effectively preventing false positives and false negatives.

[0051] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0053] Figure 1 A flowchart of a method for monitoring abnormal driving behavior provided by an embodiment of the present disclosure is shown;

[0054] Figure 2 A flowchart illustrating another abnormal driving behavior monitoring method provided by an embodiment of the present disclosure is shown;

[0055] Figure 3 A schematic diagram of a device for monitoring abnormal driving behavior provided by an embodiment of the present disclosure is shown;

[0056] Figure 4 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0058] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0059] The term "and / or" herein simply describes an association relationship, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, and the existence of B alone. In addition, the term "at least one" herein refers to any combination of at least two of any one or more of a plurality of items. For example, "at least one of A, B, and C" can represent any one or more elements selected from the set consisting of A, B, and C.

[0060] Research has found that due to resource limitations of on-board equipment, complex algorithms cannot be used during the current driver's driving state, which often leads to omissions and false alarms of some abnormal driving behaviors of the driver, such as the inability to accurately identify them. In transportation services such as hazardous chemicals, which have high requirements on the driver's driving state, if omissions and false alarms occur, the risk of safety accidents will be aggravated. Therefore, how to accurately and timely monitor the driver's driving actions at the airport and provide prompts has become an urgent problem to be solved.

[0061] Based on the above research, the present disclosure provides a method, device, and electronic device for monitoring abnormal driving behavior. The method comprises: obtaining a real-time driving video of a driver; inputting the real-time driving video into a pre-trained action recognition model; and identifying a first confidence level for each abnormal driving action performed by the driver among a plurality of preset abnormal driving actions, wherein an abnormal driving action is an action that affects driving safety; inputting the real-time driving video into a pre-trained head posture recognition model, identifying the driver's head posture, and determining a second confidence level for the head posture being a preset abnormal head posture; determining the business type of the current driving process and a preset first confidence threshold corresponding to the business type; and screening abnormal driving actions with a first confidence level greater than the preset first confidence threshold among the abnormal driving actions, and sending a corresponding abnormal action prompt message to the driver; and determining whether the second confidence level is greater than the preset first confidence threshold. If so, it is determined that the driver has experienced a distraction event, and a distraction prompt message is sent to the driver. This method can improve the sensitivity and accuracy of monitoring abnormal driving behavior of the driver, effectively preventing the occurrence of false positives and false negatives.

[0062] To facilitate understanding of this embodiment, a method for monitoring abnormal driving behavior disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the method for monitoring abnormal driving behavior provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities. The computer device includes, for example, a terminal device or a server or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method for monitoring abnormal driving behavior can be implemented by a processor calling computer-readable instructions stored in a memory.

[0063] See also Figure 1 FIG. 1 is a flow chart of a method for monitoring abnormal driving behavior provided by an embodiment of the present disclosure, the method comprising steps S101 to S105, wherein:

[0064] S101. Obtain a real-time driving video of the driver.

[0065] In a specific implementation, a surveillance camera installed inside a truck captures real-time video footage of the driver's driving process as a real-time driving video, and the real-time driving video captured by the surveillance camera is downloaded according to the video address.

[0066] Here, real-time driving video can be captured by a fixed surveillance camera installed inside the truck cab, which automatically starts shooting after the driver sets off. It can also be captured by a mobile camera distributed to the driver, which is manually placed in a fixed position in the cab after the driver gets on the vehicle and starts shooting.

[0067] It should be noted that the download address for real-time driving videos, for fixed surveillance cameras installed inside the truck cab, matches the truck's own identification code, that is, one truck corresponds to one fixed video download address; for mobile cameras issued to drivers, the download address matches the driver's employee identification code within the company, that is, one driver corresponds to one fixed video download address.

[0068] As a possible implementation, after obtaining the real-time driving video of the driver, the real-time driving video can be decoded and shortened to a preset size; the frame rate and total number of frames corresponding to the real-time driving video are determined; and according to the frame rate and the total number of frames, the real-time driving video is time-sequentially sampled to obtain multiple video frame images.

[0069] In this way, the amount of subsequent calculations for real-time driving videos can be reduced, thereby improving the response speed of monitoring abnormal driving behavior of drivers.

[0070] S102: Input the real-time driving video into a pre-trained action recognition model, and identify a first confidence level of each abnormal driving action performed by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety.

[0071] In a specific implementation, the acquired real-time driving video of the driver is input into a pre-trained action recognition model. The action recognition model is used to automatically detect the driver's character image in the real-time driving video, and identify whether the driver's current driving action is a preset abnormal driving action learned by the action recognition model during the training process, and use the confidence level as the first confidence value.

[0072] It should be noted that the action recognition model is trained based on a large amount of training video data containing human actions and corresponding action labels. The action labels here are used to mark the preset types of abnormal driving actions performed by the characters in the training video data. Optionally, the action recognition model can select a 3D convolutional neural network.

[0073] In one example, the preset abnormal driving actions may include: smoking, making phone calls, closing eyes, looking at mobile phones and other actions that may affect the driver's driving safety. They can be selected according to actual needs and are not specifically restricted here.

[0074] S103: Input the real-time driving video into a pre-trained head posture recognition model to recognize the driver's head posture, and determine a second confidence level that the head posture is a preset abnormal head posture.

[0075] In the specific implementation, the real-time driving video obtained is input into a pre-trained head posture recognition model. Since the driver's head posture can reflect the direction of the driver's attention at this time, by identifying the driver's head posture through the head posture recognition model, it is possible to monitor whether the driver's attention is on driving the truck and the road conditions ahead at this time.

[0076] Here, the head posture recognition model is used to detect whether the driver's current head posture is the same as the preset abnormal head posture, and output the confidence level that the driver's current head posture is the preset abnormal head posture as the second confidence level value.

[0077] Among them, the first confidence level and the second confidence level are values ​​less than 1, which are respectively used to represent the probability values ​​of the driver's current driving action belonging to one of the preset abnormal driving actions and the driver's current head posture being the preset abnormal head posture.

[0078] It should be noted that the recognition object of the head posture recognition model can be the movement characteristics of the driver's head or the orientation posture maintained by the driver's head within a certain period of time. Correspondingly, the same applies to the preset abnormal head posture.

[0079] In one example, the preset abnormal head posture may be that the driver's head frequently nods within a certain period of time, which indicates that the driver is likely to be fatigued and drowsy, so it can be judged that the driver is currently in a state of inattention; in another example, the preset abnormal head posture may be that the driver's head remains in a direction other than the vehicle's forward direction within a certain period of time, which indicates that the driver's attention is not currently focused on driving the vehicle and observing the road conditions ahead, so it can be judged that the driver is currently in a state of inattention.

[0080] As a possible implementation, before the real-time driving video is input into a pre-trained head posture recognition model, each of the multiple video frame images segmented from the real-time driving video can be converted into a corresponding grayscale image.

[0081] In this way, the grayscale image can better reflect the image features of the driver's head posture in the frame image, and improve the detection effect of the driver's head posture.

[0082] S104: Determine the business type of the current driving process and a preset first confidence threshold corresponding to the business type. Among the abnormal driving actions, screen target abnormal driving actions whose first confidence is greater than the preset first confidence threshold, and push corresponding abnormal action prompt information to the driver.

[0083] In practice, different driver behavior requirements vary depending on the type of cargo transported. For example, when transporting hazardous materials like dangerous chemicals, the expectation is to minimize the number of instances of driver misbehavior. Therefore, to meet the needs of different business types, different confidence thresholds are set for detecting abnormal driver behavior and distracted driving, tailored to the specific type of driving process.

[0084] Specifically, among the multiple preset abnormal driving actions identified by the action recognition model, the target abnormal driving actions whose corresponding confidence levels are greater than a preset first confidence threshold are screened out, that is, it is considered that the recognition success rate of the target abnormal driving actions is higher, which means that the probability of the driver performing the target abnormal driving actions is higher.

[0085] For example, during the transportation of hazardous chemicals, the action recognition model identified three abnormal driving actions of the driver: looking at the mobile phone, smoking, and making a phone call, and the corresponding first confidence levels were 0.8, 0.2, and 0.5. If the preset first confidence threshold corresponding to the hazardous chemical business type is 0.6, then the two abnormal driving actions of smoking and making a phone call are likely to be false alarms. Therefore, only "looking at the mobile phone" is used as the target abnormal driving action.

[0086] As a possible implementation method, abnormal action prompt information corresponding to the screened target abnormal driving action can be pushed to the vehicle terminal to prompt the driver to make corrections in a timely manner. The abnormal action prompt information can be text or voice information. For example, when the target abnormal driving action is: the driver looking at the phone while driving, the abnormal action prompt information can be "It is detected that you are looking at the phone while driving, please correct it in time." Furthermore, the abnormal action prompt information can be displayed in the form of text on the vehicle terminal. If the driver's attention may be distracted by looking at the abnormal action prompt information while driving, the abnormal action prompt information can be read aloud by voice on the vehicle terminal to prompt the driver to make corrections in a timely manner.

[0087] S105: Determine whether the second confidence level is greater than the preset first confidence level threshold; if so, determine that the driver has experienced a distraction event, and push distraction prompt information to the driver.

[0088] In a specific implementation, the second confidence level output by the head posture recognition model is compared with a preset first confidence level threshold. If the second confidence level is greater than the preset first confidence level threshold, it can be confirmed that in the real-time driving video, the driver's head posture matches the abnormal head posture preset in the head posture recognition model. At this time, it can be determined that the driver has experienced an incident of distraction.

[0089] For example, the head posture recognition model detects that the driver has frequently nodded his head over a period of time, and the corresponding second confidence level is 0.8. If the first confidence level threshold is preset to 0.6 at this time, it can be confirmed that the recognition is successful. The driver is more likely to have frequently nodded his head, and the probability that the driver is distracted is also higher. Therefore, a distraction prompt message is sent to the driver to remind the driver to focus on driving the vehicle and observing the road conditions ahead.

[0090] The disclosed embodiments provide a method for monitoring abnormal driving behavior. The method comprises obtaining a real-time driving video of a driver; inputting the real-time driving video into a pre-trained action recognition model, and identifying a first confidence level for each abnormal driving action performed by the driver among a plurality of preset abnormal driving actions, wherein an abnormal driving action is an action that affects driving safety; inputting the real-time driving video into a pre-trained head posture recognition model, identifying the driver's head posture, and determining a second confidence level for the head posture being a preset abnormal head posture; determining a business type of the current driving process and a preset first confidence threshold corresponding to the business type; screening abnormal driving actions with a first confidence level greater than the preset first confidence threshold among the abnormal driving actions, and sending a corresponding abnormal action prompt message to the driver; and determining whether the second confidence level is greater than the preset first confidence threshold. If so, determining that the driver has experienced a distraction event and sending a distraction prompt message to the driver. The method can improve the sensitivity and accuracy of monitoring abnormal driving behavior of the driver and effectively prevent the occurrence of false positives and false negatives.

[0091] See also Figure 2 FIG. 2 is a flow chart of another abnormal driving behavior monitoring method provided by an embodiment of the present disclosure, wherein the method includes steps S201 to S208, wherein:

[0092] S201: Obtain the driver's real-time driving video.

[0093] S202: Input the real-time driving video into a pre-trained action recognition model, and identify a first confidence level of each abnormal driving action performed by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety.

[0094] S203: Input the real-time driving video into a pre-trained head posture recognition model to recognize the driver's head posture, and determine a second confidence level that the head posture is a preset abnormal head posture.

[0095] S204: Determine the business type of the current driving process and a preset first confidence threshold corresponding to the business type. Among the abnormal driving actions, screen target abnormal driving actions whose first confidence is greater than the preset first confidence threshold, and push corresponding abnormal action prompt information to the driver.

[0096] S205. Determine a preset second confidence threshold corresponding to the business type, wherein the preset second confidence threshold is less than the preset first confidence threshold; and among the abnormal driving actions, screen the abnormal driving actions to be confirmed whose first confidence is within the interval between the preset first confidence threshold and the preset second confidence threshold.

[0097] In a specific implementation, a corresponding second confidence threshold is set according to the business type, wherein the second confidence threshold is the minimum confidence threshold reflecting the successful identification of abnormal driving actions and distraction events. That is to say, if the first confidence and the second confidence fall within the interval between the first confidence threshold and the second confidence threshold, it is considered impossible to accurately determine whether the driver has made abnormal driving actions. Therefore, the abnormal driving actions corresponding to the identified first confidence within the interval between the first confidence threshold and the second confidence threshold are set as abnormal driving actions to be confirmed, which require manual intervention for identification to prevent the occurrence of missed reports.

[0098] It should be noted that the second confidence threshold can be set according to actual needs and is not specifically limited here.

[0099] S206: Send the abnormal driving action to be confirmed to a preset manual confirmation platform, and receive a confirmation result corresponding to the abnormal driving action to be confirmed returned by the manual confirmation platform, where the confirmation result includes a normal action and an abnormal action; filter out the abnormal driving action to be confirmed whose confirmation result is an abnormal action, and push corresponding abnormal action prompt information to the driver.

[0100] In the specific implementation, manual identification is performed on the artificial platform for the abnormal driving actions to be confirmed to determine whether the driver actually has the abnormal driving actions to be confirmed in the real-time driving video. If the result of the manual identification is that the abnormal action is confirmed, the corresponding abnormal action prompt information is pushed to the driver; further, if the result of the manual identification is that the driver does not have the above-mentioned abnormal driving action to be confirmed, the abnormal action prompt information will not be pushed.

[0101] Furthermore, if the first confidence level is less than a preset second confidence level threshold, it is directly determined that there is no abnormal driving action by the driver in the real-time driving video, and the abnormal action prompt information will not be pushed without the intervention of the manual platform.

[0102] In this way, for abnormal driving actions that cannot be confirmed by automatic recognition, the manual platform intervenes to perform manual confirmation to prevent the omission of abnormal driving actions of the driver due to recognition errors of the action recognition model.

[0103] S207: Determine whether the second confidence level is greater than the preset first confidence level threshold; if so, determine that the driver has experienced a distraction event, and push distraction prompt information to the driver.

[0104] It should be noted that steps S201 to S204 and step S207 are substantially the same as steps S101 to S105, have the same implementation methods and can achieve the same technical effects, and will not be described in detail here.

[0105] S208. If the second confidence level is less than the preset first confidence level threshold, determine whether the second confidence level is greater than the preset second confidence level threshold; if it is, send the driver's head posture to the manual confirmation platform, and receive the attention distraction confirmation result returned by the manual confirmation platform, wherein the attention distraction confirmation result includes attention distraction and attention non-distraction; for the head posture whose confirmation result is attention distraction, push attention distraction prompt information to the driver.

[0106] In a specific implementation, the abnormal head posture corresponding to the second confidence level identified to be within the interval between the first confidence threshold and the second confidence threshold is set to a pending confirmation state, which requires manual intervention to prevent missed reports.

[0107] Here, manual identification is performed on the abnormal head posture in the state to be confirmed through an artificial platform to determine whether the driver is actually distracted by the abnormal head posture in the real-time driving video. If the result of the manual identification is that there is an abnormal head posture that may cause distraction, the attention distraction prompt information is pushed to the driver; further, if the result of the manual identification is that the driver does not have an abnormal head posture that may cause distraction, the attention distraction prompt information is not pushed.

[0108] Furthermore, if the second confidence level is less than a preset second confidence level threshold, it is directly determined that the driver in the real-time driving video does not have an abnormal head posture that may cause distraction, and the distraction prompt information will not be pushed without the intervention of the manual platform.

[0109] In this way, when automatic recognition cannot confirm whether there is an attention distraction event, the manual platform intervenes to perform manual confirmation to prevent the underreporting of driver attention distraction events due to recognition errors of the head posture recognition model.

[0110] The disclosed embodiments provide a method for monitoring abnormal driving behavior. The method comprises obtaining a real-time driving video of a driver; inputting the real-time driving video into a pre-trained action recognition model, and identifying a first confidence level for each abnormal driving action performed by the driver among a plurality of preset abnormal driving actions, wherein an abnormal driving action is an action that affects driving safety; inputting the real-time driving video into a pre-trained head posture recognition model, identifying the driver's head posture, and determining a second confidence level for the head posture being a preset abnormal head posture; determining a business type of the current driving process and a preset first confidence threshold corresponding to the business type; screening abnormal driving actions with a first confidence level greater than the preset first confidence threshold among the abnormal driving actions, and sending a corresponding abnormal action prompt message to the driver; and determining whether the second confidence level is greater than the preset first confidence threshold. If so, determining that the driver has experienced a distraction event and sending a distraction prompt message to the driver. The method can improve the sensitivity and accuracy of monitoring abnormal driving behavior of the driver and effectively prevent the occurrence of false positives and false negatives. Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0111] Based on the same inventive concept, the embodiments of the present disclosure also provide a device for monitoring abnormal driving behavior corresponding to the method for monitoring abnormal driving behavior. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the above-mentioned method for monitoring abnormal driving behavior in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0112] See also Figure 3 , Figure 3 Schematic diagram of a monitoring device for abnormal driving behavior provided by an embodiment of the present disclosure. Figure 3 As shown in , the monitoring device 300 provided in the embodiment of the present disclosure includes:

[0113] The video acquisition module 310 is used to acquire the real-time driving video of the driver;

[0114] The action recognition module 320 is configured to input the real-time driving video into a pre-trained action recognition model and identify a first confidence level of each abnormal driving action by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety;

[0115] a posture recognition module 330 for inputting the real-time driving video into a pre-trained head posture recognition model, identifying the driver's head posture, and determining a second confidence level that the head posture is a preset abnormal head posture;

[0116] The abnormal action prompt module 340 is configured to determine the business type of the current driving process and a preset first confidence threshold corresponding to the business type, select target abnormal driving actions whose first confidence is greater than the preset first confidence threshold among the abnormal driving actions, and push corresponding abnormal action prompt information to the driver;

[0117] The attention distraction prompt module 350 is used to determine whether the second confidence level is greater than the preset first confidence level threshold. If so, it is determined that the driver has an attention distraction event and pushes attention distraction prompt information to the driver.

[0118] In an optional embodiment, the monitoring device 300 further includes an abnormal action verification module, and the abnormal action verification module is specifically configured to:

[0119] Determining a preset second confidence threshold corresponding to the service type, wherein the preset second confidence threshold is less than the preset first confidence threshold;

[0120] Among the abnormal driving actions, screening the abnormal driving actions to be confirmed whose first confidence level is within the interval between the preset first confidence threshold and the preset second confidence threshold;

[0121] Sending the abnormal driving action to be confirmed to a preset manual confirmation platform, and receiving a confirmation result corresponding to the abnormal driving action to be confirmed returned by the manual confirmation platform, wherein the confirmation result includes a normal action and an abnormal action;

[0122] The abnormal driving actions to be confirmed whose confirmation results are abnormal actions are screened out, and corresponding abnormal action prompt information is pushed to the driver.

[0123] In an optional embodiment, the monitoring device 300 further includes an attention distraction verification module, and the attention distraction verification module is specifically configured to:

[0124] If the second confidence level is less than the preset first confidence level threshold, determining whether the second confidence level is greater than the preset second confidence level threshold;

[0125] If it is greater than, sending the driver's head posture to the manual confirmation platform, and receiving the attention distraction confirmation result returned by the manual confirmation platform, wherein the attention distraction confirmation result includes attention distraction and attention non-distraction;

[0126] For the head posture whose confirmation result is distraction, distraction prompt information is pushed to the driver.

[0127] In an optional embodiment, the monitoring device 300 further includes a video preprocessing module, which is configured to:

[0128] decoding the real-time driving video, and shortening the real-time driving video to a preset size;

[0129] Determining a frame rate and a total number of frames corresponding to the real-time driving video;

[0130] According to the frame rate and the total number of frames, time-sequential sampling is performed on the real-time driving video to obtain a plurality of video frame images.

[0131] In an optional implementation, the action recognition module is further configured to:

[0132] For each of the video frame images, the video frame image is converted into a corresponding grayscale image.

[0133] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0134] The present disclosure provides a device for monitoring abnormal driving behavior. The device obtains a real-time driving video of a driver; inputs the real-time driving video into a pre-trained action recognition model, and identifies a first confidence level for each abnormal driving action performed by the driver among a plurality of preset abnormal driving actions, wherein an abnormal driving action is an action that affects driving safety; inputs the real-time driving video into a pre-trained head posture recognition model, identifies the driver's head posture, and determines a second confidence level for the head posture being a preset abnormal head posture; determines the business type of the current driving process and a preset first confidence threshold corresponding to the business type; among the abnormal driving actions, screens for target abnormal driving actions with a first confidence level greater than the preset first confidence threshold, and pushes corresponding abnormal action prompt information to the driver; and determines whether the second confidence level is greater than the preset first confidence threshold. If so, it is determined that the driver has experienced a distraction event, and pushes a distraction prompt information to the driver. This device can improve the sensitivity and accuracy of monitoring abnormal driving behavior of the driver, effectively preventing the occurrence of false positives and false negatives.

[0135] Corresponding to Figure 1 The present disclosure also provides an electronic device 400, such as a monitoring method for abnormal driving behavior. Figure 4 FIG. 4 is a schematic diagram of the structure of an electronic device 400 provided in an embodiment of the present disclosure, including:

[0136] Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including memory 421 and external memory 422; the memory 421 here is also called internal memory, which is used to temporarily store the operation data in the processor 41 and the data exchanged with the external memory 422 such as the hard disk. The processor 41 exchanges data with the external memory 422 through the memory 421. When the electronic device 400 is running, the processor 41 and the memory 42 communicate through the bus 43, so that the processor 41 executes Figure 1 and Figure 2 Steps of a method for monitoring abnormal driving behavior.

[0137] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the abnormal driving behavior monitoring method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0138] The embodiments of the present disclosure also provide a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, the steps of the abnormal driving behavior monitoring method described in the above method embodiment can be executed. For details, please refer to the above method embodiment, which will not be repeated here.

[0139] The computer program product may be implemented in hardware, software, or a combination thereof. In one embodiment, the computer program product is implemented as a computer storage medium. In another embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0142] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0144] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A method for monitoring abnormal driving behavior, characterized in that: The method comprises: Get real-time driving video of the driver; Inputting the real-time driving video into a pre-trained action recognition model, and identifying a first confidence level of each abnormal driving action by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety; Inputting the real-time driving video into a pre-trained head posture recognition model to recognize the driver's head posture, and determining a second confidence level that the head posture is a preset abnormal head posture; determining a business type of the current driving process and a preset first confidence threshold corresponding to the business type, screening target abnormal driving actions whose first confidence is greater than the preset first confidence threshold among the abnormal driving actions, and pushing corresponding abnormal action prompt information to the driver; determining whether the second confidence level is greater than the preset first confidence level threshold; if so, determining that the driver has experienced a distraction event, and pushing a distraction prompt message to the driver; After determining the business type of the current driving process and the preset first confidence threshold corresponding to the business type, screening target abnormal driving actions having the first confidence greater than the preset first confidence threshold among the abnormal driving actions, and pushing abnormal action prompt information to the driver, the method further includes: Determining a preset second confidence threshold corresponding to the service type, wherein the preset second confidence threshold is less than the preset first confidence threshold; Among the abnormal driving actions, screening the abnormal driving actions to be confirmed whose first confidence level is within the interval between the preset first confidence threshold and the preset second confidence threshold; Sending the abnormal driving action to be confirmed to a preset manual confirmation platform, and receiving a confirmation result corresponding to the abnormal driving action to be confirmed returned by the manual confirmation platform, wherein the confirmation result includes a normal action and an abnormal action; The abnormal driving actions to be confirmed whose confirmation results are abnormal actions are screened out, and corresponding abnormal action prompt information is pushed to the driver.

2. The method according to claim 1, characterized in that After determining whether the second confidence level is greater than the preset first confidence level threshold, and if so, determining that the driver has experienced a distraction event and pushing distraction prompt information to the driver, the method further includes: If the second confidence level is less than the preset first confidence level threshold, determining whether the second confidence level is greater than the preset second confidence level threshold; If it is greater than, sending the driver's head posture to the manual confirmation platform, and receiving the attention distraction confirmation result returned by the manual confirmation platform, wherein the attention distraction confirmation result includes attention distraction and attention non-distraction; For the head posture whose confirmation result is distraction, distraction prompt information is pushed to the driver.

3. The method according to claim 1, characterized in that After obtaining the real-time driving video of the driver, the method further includes: decoding the real-time driving video, and shortening the real-time driving video to a preset size; Determining a frame rate and a total number of frames corresponding to the real-time driving video; According to the frame rate and the total number of frames, time-sequential sampling is performed on the real-time driving video to obtain a plurality of video frame images.

4. The method according to claim 3, characterized in that Before inputting the real-time driving video into a pre-trained head posture recognition model to recognize the driver's head posture and determining that the head posture is a preset abnormal head posture with a second confidence level, the method further includes: For each of the video frame images, the video frame image is converted into a corresponding grayscale image.

5. The method according to claim 1, wherein: The action recognition model is a 3D convolutional neural network.

6. A monitoring device for abnormal driving behavior, characterized in that: include: Video acquisition module, used to obtain real-time driving video of the driver; a motion recognition module, configured to input the real-time driving video into a pre-trained motion recognition model and identify a first confidence level of each abnormal driving action by the driver among a plurality of preset abnormal driving actions, wherein the abnormal driving action is an action that affects driving safety; a posture recognition module, configured to input the real-time driving video into a pre-trained head posture recognition model, recognize the driver's head posture, and determine a second confidence level that the head posture is a preset abnormal head posture; an abnormal action prompting module, configured to determine a business type of a current driving process and a preset first confidence threshold corresponding to the business type, screen target abnormal driving actions whose first confidence is greater than the preset first confidence threshold among the abnormal driving actions, and push corresponding abnormal action prompt information to the driver; an attention distraction prompting module, configured to determine whether the second confidence level is greater than the preset first confidence level threshold, and if so, determine that the driver has experienced an attention distraction event, and push an attention distraction prompting message to the driver; The device further includes an abnormal action verification module, which is specifically configured to: Determining a preset second confidence threshold corresponding to the service type, wherein the preset second confidence threshold is less than the preset first confidence threshold; Among the abnormal driving actions, screening the abnormal driving actions to be confirmed whose first confidence level is within the interval between the preset first confidence threshold and the preset second confidence threshold; Sending the abnormal driving action to be confirmed to a preset manual confirmation platform, and receiving a confirmation result corresponding to the abnormal driving action to be confirmed returned by the manual confirmation platform, wherein the confirmation result includes a normal action and an abnormal action; The abnormal driving actions to be confirmed whose confirmation results are abnormal actions are screened out, and corresponding abnormal action prompt information is pushed to the driver.

7. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the abnormal driving behavior monitoring method as described in any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the abnormal driving behavior monitoring method according to any one of claims 1 to 5.

Images