Method, device and equipment for detecting inattentive driving behavior and storage medium

Abstract

The present application relates to the technical field of image recognition, and discloses a detection method, device and equipment for distracted driving behavior and a storage medium, which are used to improve the recognition accuracy of distracted driving behavior. The detection method for distracted driving behavior comprises the following steps: obtaining initial driving video data of a driver, performing data processing on the initial driving video data, and obtaining to-be-recognized behavior video data; calling a preset distracted driving behavior model to recognize the to-be-recognized behavior video data; if the driver has candidate distracted driving behavior, performing special behavior detection on the driver and vehicle driving condition detection on a target vehicle in which the driver is located; and if the driver has no special behavior and the target vehicle is in a normal vehicle driving condition, it is determined that the driver has target distracted driving behavior.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method, apparatus, device, and storage medium for detecting distracted driving behavior. Background Technology

[0002] As society continues to develop, the demand for and use of cars are gradually increasing. Various behaviors that affect traffic safety will always occur when driving. Among them, distracted or fatigued driving is particularly common. Once the driver is distracted while driving, it is very easy to cause traffic accidents.

[0003] In recent years, research on distracted driving behavior recognition based on convolutional neural networks has made some progress, but existing algorithm models have large weight files and long computation time, resulting in low recognition accuracy. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for detecting distracted driving behavior, which improves the accuracy of identifying distracted driving behavior.

[0005] The first aspect of this invention provides a method for detecting distracted driving behavior, comprising: acquiring initial driving video data of a driver, and processing the initial driving video data to obtain video data of behavior to be identified; and calling a preset distracted driving behavior model to identify the video data of behavior to be identified, wherein the feature mapping of the distracted driving behavior model is represented as A∈R N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time. If the driver exhibits candidate distracted driving behavior, special behavior detection is performed on the driver, and vehicle driving status detection is performed on the target vehicle where the driver is located. If the driver does not exhibit special behavior and the target vehicle is in normal driving status, then it is determined that the driver exhibits target distracted driving behavior.

[0006] In one feasible implementation, the step of acquiring the driver's initial driving video data and processing the initial driving video data to obtain video data of the behavior to be identified includes: acquiring the driver's initial driving video data and determining whether the video length of the initial driving video data is greater than or equal to a preset length; if it is greater than or equal to the preset length, sampling the initial driving video data based on the preset length to obtain first candidate driving video data; performing data enhancement processing on the first candidate driving video data to generate video data of the behavior to be identified, wherein the data enhancement processing includes mirror flipping, video reversal, video cropping, and / or video splicing.

[0007] In one feasible implementation, before acquiring the driver's initial driving video data and processing the initial driving video data to obtain the behavior video data to be identified, the method further includes: acquiring the driver's historical driving video data and training a preset model based on the historical driving video data to generate a distracted driving behavior model, wherein the historical driving video data includes multiple distracted driving behaviors of the driver.

[0008] In one feasible implementation, the step of acquiring the driver's historical driving video data and training a preset model based on the historical driving video data to generate a distracted driving behavior model includes: acquiring the driver's historical driving video data and classifying multiple distracted driving behaviors in the historical driving video data to obtain multiple distracted driving behavior type video data; setting the preset model based on preset parameters to obtain a set preset model; inserting the set preset model into a residual network and performing data augmentation processing to generate a candidate model; and training the candidate model based on the multiple distracted driving behavior type video data to generate a distracted driving behavior model.

[0009] In one feasible implementation, the step of detecting special behaviors of the driver and detecting the driving status of the target vehicle where the driver is located if the driver exhibits candidate distracted driving behavior includes: generating a behavior feature map of the driver based on the video data of the behavior to be identified if the driver exhibits candidate distracted driving behavior; determining that the driver does not exhibit special behavior if the behavior feature map does not match the corresponding preset special behavior feature map; and detecting the driving status of the target vehicle where the driver is located.

[0010] In one feasible implementation, the vehicle driving status detection of the target vehicle where the driver is located includes: acquiring an image of the road ahead and an image of the windshield of the target vehicle where the driver is located; performing road detection on the road ahead image and identifying obstructions on the windshield image; if the target vehicle meets preset conditions, then determining that the target vehicle is in a normal driving condition, wherein the preset conditions are that the road ahead in the road ahead image is open to traffic and there are no obstructions on the windshield in the windshield image.

[0011] In one feasible implementation, after detecting the driver's special behavior and the vehicle driving status of the target vehicle where the driver is located if the driver exhibits candidate distracted driving behavior, the method further includes: if the driver exhibits special behavior related to a sudden illness, generating vehicle location information, sudden illness information, and vehicle control information for the target vehicle where the driver is located, wherein the vehicle control information is used to control the target vehicle to stop and turn on the corresponding vehicle lights; and sending the vehicle location information and the sudden illness information to a preset rescue platform.

[0012] A second aspect of the present invention provides a device for detecting distracted driving behavior, comprising: an acquisition and processing module for acquiring initial driving video data of a driver and processing the initial driving video data to obtain video data of a behavior to be identified; and an invocation and identification module for invoking a preset distracted driving behavior model to identify the video data of the behavior to be identified, wherein the feature mapping of the distracted driving behavior model is represented as A∈R N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time. The detection module is used to detect special behaviors of the driver and the vehicle driving status of the target vehicle where the driver is located if the driver has candidate distracted driving behavior. The determination module is used to determine that the driver has target distracted driving behavior if the driver does not have special behavior and the target vehicle is in normal driving status.

[0013] In one feasible implementation, the acquisition and processing module is specifically used to: acquire initial driving video data of the driver, and determine whether the video length of the initial driving video data is greater than or equal to a preset length; if it is greater than or equal to the preset length, sample the initial driving video data based on the preset length to obtain first candidate driving video data; perform data enhancement processing on the first candidate driving video data to generate video data of the behavior to be identified, wherein the data enhancement processing includes mirror flipping, video reversal, video cropping and / or video splicing.

[0014] In one feasible implementation, the distracted driving behavior detection device further includes: an acquisition and generation module, used to acquire the driver's historical driving video data, and train a preset model based on the historical driving video data to generate a distracted driving behavior model, wherein the historical driving video data includes multiple distracted driving behaviors of the driver.

[0015] In one feasible implementation, the acquisition and generation module is specifically used for: acquiring the driver's historical driving video data, classifying multiple distracted driving behaviors in the historical driving video data to obtain multiple distracted driving behavior type video data; setting a preset model based on preset parameters to obtain a set preset model; inserting the set preset model into a residual network and performing data augmentation processing to generate a candidate model; and training the candidate model based on the multiple distracted driving behavior type video data to generate a distracted driving behavior model.

[0016] In one feasible implementation, the detection module includes: a feature map generation unit, configured to generate a behavior feature map of the driver based on the video data of the behavior to be identified if the driver has candidate distracted driving behavior; a determination unit, configured to determine that the driver does not have special behavior if the behavior feature map does not match the corresponding preset special behavior feature map; and a vehicle detection unit, configured to detect the driving status of the target vehicle where the driver is located.

[0017] In one feasible implementation, the vehicle detection unit is specifically used to: acquire an image of the road ahead and an image of the windshield of the target vehicle where the driver is located; perform road detection on the road ahead image and identify obstructions on the windshield image; if the target vehicle meets preset conditions, determine that the target vehicle is in a normal driving condition, wherein the preset conditions are that the road ahead in the road ahead image is open to traffic and there are no obstructions on the windshield in the windshield image.

[0018] In one feasible implementation, the distracted driving behavior detection device further includes: an information generation module, used to generate vehicle location information, sudden illness information, and vehicle control information of the target vehicle where the driver is located if the driver exhibits a special behavior of sudden illness, wherein the vehicle control information is used to control the target vehicle to stop and turn on the corresponding vehicle lights; and an information sending module, used to send the vehicle location information and the sudden illness information to a preset rescue platform.

[0019] A third aspect of the present invention provides a device for detecting distracted driving behavior, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the device for detecting distracted driving behavior to perform the above-described method for detecting distracted driving behavior.

[0020] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the above-described method for detecting distracted driving behavior.

[0021] The technical solution provided by this invention involves acquiring initial driving video data of the driver, processing the initial driving video data to obtain video data of the behavior to be identified, and then calling a preset distracted driving behavior model to identify the video data of the behavior to be identified. The feature mapping of the distracted driving behavior model is represented as A∈R. N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model shifts channels by ±1 in the time dimension, moving 1 / 4 of the total number of channels each time. If the driver exhibits candidate distracted driving behavior, special behavior detection is performed on the driver, and vehicle driving status detection is performed on the target vehicle where the driver is located. If the driver does not exhibit special behavior and the target vehicle is in normal driving status, then the driver is determined to have target distracted driving behavior. In this embodiment of the invention, the driver's behavior video data to be identified is identified through a pre-set distracted driving behavior model. If the driver does not exhibit special behavior and the target vehicle is in normal driving status, then the driver is determined to have target distracted driving behavior, reducing the computation time for behavior recognition, achieving lightweight distracted driving behavior recognition, and eliminating misjudgments of distracted driving behavior caused by the driver's special behavior and vehicle driving status, thereby improving the accuracy of distracted driving behavior recognition. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of one embodiment of the method for detecting distracted driving behavior in this invention.

[0023] Figure 2 This is a schematic diagram of another embodiment of the distracted driving behavior detection method in this invention;

[0024] Figure 3 This is a schematic diagram of one embodiment of the distracted driving behavior detection device in this invention;

[0025] Figure 4 This is a schematic diagram of another embodiment of the distracted driving behavior detection device in this invention;

[0026] Figure 5 This is a schematic diagram of one embodiment of the distracted driving behavior detection device in this invention. Detailed Implementation

[0027] This invention provides a method, apparatus, device, and storage medium for detecting distracted driving behavior, which improves the accuracy of identifying distracted driving behavior.

[0028] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0030] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0031] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the method for detecting distracted driving behavior in this invention includes:

[0032] 101. Obtain the driver's initial driving video data and process the initial driving video data to obtain the video data of the behavior to be identified;

[0033] It is understood that the executing entity of this invention can be a device for detecting distracted driving behavior, or a device for detecting distracted driving behavior; the specific implementation is not limited here. This embodiment of the invention will be described using a device for detecting distracted driving behavior as an example.

[0034] The initial driving video data includes two types of driving behavior: attentive driving behavior and distracted driving behavior. For example, attentive driving behavior includes keeping both hands on the steering wheel and looking at the road ahead, not making or receiving phone calls, not smoking, not picking up items, not drinking water, not swaying from side to side, and not looking around. Distracted driving behavior includes taking both hands off the steering wheel, making or receiving phone calls, smoking, picking up items, drinking water, swaying from side to side, looking around, and eating.

[0035] 102. Call the pre-set distracted driving behavior model to identify the video data of the behavior to be identified. The feature mapping of the distracted driving behavior model is represented as A∈R. N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time.

[0036] The distracted driving behavior model used in this embodiment is the Temporal Shift Module (TSM), which promotes information exchange between adjacent frames by shifting some channels along the time dimension. In this embodiment, the channels are shifted by ±1 in the time dimension, which reduces the computation time of behavior recognition and realizes lightweight distracted driving behavior recognition.

[0037] 103. If the driver exhibits candidate distracted driving behavior, special behavior detection will be performed on the driver, and vehicle driving status detection will be performed on the target vehicle where the driver is located.

[0038] For example, special behaviors include, but are not limited to, a driver taking both hands off the steering wheel due to a sudden illness, a driver looking around due to obstructed vision caused by an obstruction in the windshield of the target vehicle, or a driver drinking water or eating when the road in front of the target vehicle is closed to traffic. The vehicle's driving conditions include, but are not limited to, whether there is an obstruction in the windshield of the target vehicle, and whether the road in front of the vehicle is open to traffic.

[0039] For example, if a driver exhibits distracted driving behavior such as taking both hands off the steering wheel, the distracted driving behavior detection device will perform special behavior detection on the driver and vehicle driving status detection on the target vehicle where the driver is located.

[0040] 104. If the driver does not exhibit any special behavior and the target vehicle is in normal driving condition, then it is determined that the driver is engaging in distracted driving behavior.

[0041] For example, based on the example in step 103, if the driver does not have a sudden illness that causes his hands to leave the steering wheel, the road in front of the target vehicle is open and there are no obstructions on the windshield, then the distracted driving behavior detection device determines that the driver has the target distracted driving behavior.

[0042] In this embodiment of the invention, a pre-set distracted driving behavior model is used to identify the video data of the driver's behavior to be identified. If the driver does not exhibit any special behavior and the target vehicle is in a normal driving condition, it is determined that the driver exhibits the target distracted driving behavior. This reduces the computation time for behavior recognition, achieves lightweight distracted driving behavior recognition, and eliminates the possibility of misjudgment of distracted driving behavior caused by the driver's special behavior and the vehicle's driving condition, thereby improving the accuracy of distracted driving behavior recognition.

[0043] Please see Figure 2 Another embodiment of the method for detecting distracted driving behavior in this invention includes:

[0044] 201. Obtain the driver's historical driving video data, and train the preset model based on the historical driving video data to generate a distracted driving behavior model. The historical driving video data includes multiple distracted driving behaviors of the driver.

[0045] Specifically, (1) the distracted driving behavior detection device acquires the driver's historical driving video data and classifies multiple distracted driving behaviors in the historical driving video data to obtain multiple distracted driving behavior type video data; (2) the distracted driving behavior detection device sets the preset model based on preset parameters to obtain the preset model after setting; (3) the distracted driving behavior detection device inserts the preset model after setting into the residual network and performs data augmentation processing to generate candidate models; (4) the distracted driving behavior detection device trains the candidate models based on multiple distracted driving behavior type video data to generate a distracted driving behavior model.

[0046] The preset parameters include shifting the channels by ±1 in the time dimension and shifting 1 / 4 of the total number of channels each time.

[0047] This embodiment is based on the framework of Convolutional Neural Networks (CNN), but other neural networks can also be used, which is not limited here. This embodiment uses a Residual Network (ResNet) as the backbone network of the distracted driving behavior model. The preset model is inserted into the residual block of the Residual Network to become a variable residual block. Then, the variable residual block is used as the base block to build the residual network model and perform data augmentation processing to generate candidate models. Other backbone networks can also be used in this embodiment, which is not limited here.

[0048] This embodiment uses convolutional neural network regularization (Cutout) and mixed-up augmentation for data augmentation. Other data augmentation methods can also be used, which are not limited here.

[0049] For example, a distracted driving behavior detection device acquires historical driving video data of the driver and classifies multiple distracted driving behaviors in the historical driving video data to obtain multiple types of distracted driving behavior video data, namely: video data of hands off the steering wheel, video data of making or receiving phone calls, video data of smoking, and video data of picking up objects. The distracted driving behavior detection device sets a preset model based on preset parameters to obtain the set preset model. The preset parameters include shifting the channels by ±1 in the time dimension and shifting 1 / 4 of the number of all channels each time. The distracted driving behavior detection device inserts the set preset model into a residual network and performs data augmentation processing to generate candidate models. The distracted driving behavior detection device trains the candidate models based on the video data of multiple types of distracted driving behavior to generate a distracted driving behavior model.

[0050] 202. Obtain the driver's initial driving video data and process the initial driving video data to obtain the video data of the behavior to be identified;

[0051] Specifically, (1) the distracted driving behavior detection device acquires the driver's initial driving video data and determines whether the video length of the initial driving video data is greater than or equal to a preset length; (2) if it is greater than or equal to the preset length, the distracted driving behavior detection device samples the initial driving video data based on the preset length to obtain the first candidate driving video data; (3) the distracted driving behavior detection device performs data enhancement processing on the first candidate driving video data to generate video data of the behavior to be identified. The data enhancement processing includes mirror flipping, video reversal, video cropping and / or video splicing.

[0052] The preset length can be one minute or two minutes, and the specific preset length can be set according to the actual application scenario.

[0053] For example, with a preset length of three minutes, the distracted driving behavior detection device acquires the driver's initial driving video data and determines whether the video length of the initial driving video data is greater than or equal to three minutes. If it is greater than or equal to three minutes, the distracted driving behavior detection device samples the initial driving video data based on the preset length to obtain first candidate driving video data. The first candidate driving video data includes one or more driving video data of preset length. The distracted driving behavior detection device performs data augmentation processing on the first candidate driving video data to generate video data of the behavior to be identified. The data augmentation processing includes mirror flipping, video reversal, video cropping, and / or video splicing.

[0054] 203. Call the pre-set distracted driving behavior model to identify the video data of the behavior to be identified. The feature mapping of the distracted driving behavior model is represented as A∈R N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time.

[0055] The distracted driving behavior model fuses information from adjacent frames with the current frame when movement ends. The pre-defined convolution formula for this fusion process is: Y = w1X -1 +w2X 0 +w3X +1 Where X is the input, X i For the i-th input, X i The input X moves by -1, 0, or +1, and w1 is the value corresponding to X. -1 The weights, w2 is X 0 The weights, w3 is X +1 The weight.

[0056] 204. If the driver exhibits candidate distracted driving behavior, then special behavior detection will be performed on the driver, and vehicle driving status detection will be performed on the target vehicle where the driver is located.

[0057] Specifically, 1) if the driver exhibits candidate distracted driving behavior, the distracted driving behavior detection device generates a driver behavior feature map based on the video data of the behavior to be identified; 2) if the behavior feature map does not match the corresponding preset special behavior feature map, the distracted driving behavior detection device determines that the driver does not exhibit any special behavior; 3) the distracted driving behavior detection device detects the vehicle driving status of the target vehicle where the driver is located.

[0058] Among them, 3) the distracted driving behavior detection device detects the driving status of the target vehicle where the driver is located, specifically including: (1) the distracted driving behavior detection device acquires the image of the road ahead and the image of the windshield of the target vehicle where the driver is located; (2) the distracted driving behavior detection device performs road detection on the image of the road ahead and identifies obstructions on the image of the windshield; (3) if the target vehicle meets the preset conditions, the distracted driving behavior detection device determines that the target vehicle is in a normal driving condition, the preset conditions being that the road ahead in the image of the road ahead is in a passable state and there are no obstructions on the windshield in the image of the windshield.

[0059] Preset special behavioral feature maps include, but are not limited to, behavioral feature maps of drivers taking their hands off the steering wheel due to sudden illness, and behavioral feature maps of drivers looking left and right due to obstructions to their vision caused by the windshield of the target vehicle.

[0060] For example, if a driver exhibits a candidate distracted driving behavior of taking both hands off the steering wheel, the distracted driving behavior detection device generates a behavioral feature map of the driver taking both hands off the steering wheel based on the video data of the behavior to be identified. If the behavioral feature map of taking both hands off the steering wheel does not match the behavioral feature map of the driver taking both hands off the steering wheel due to a sudden illness, the distracted driving behavior detection device determines that the driver has not exhibited any special behavior. The distracted driving behavior detection device acquires images of the road ahead and the windshield of the target vehicle where the driver is located. The distracted driving behavior detection device performs road detection on the road ahead image and identifies obstructions in the windshield image. If the target vehicle meets preset conditions, the distracted driving behavior detection device determines that the target vehicle is in a normal driving condition. The preset conditions are that the road ahead in the road ahead image is open to traffic and there are no obstructions on the windshield in the windshield image.

[0061] 205. If the driver does not exhibit any unusual behavior and the target vehicle is in normal driving condition, then it is determined that the driver is engaging in distracted driving behavior.

[0062] For example, if the driver does not exhibit any special behavior of looking around due to obstructed vision caused by an obstruction in the windshield of the target vehicle, and the windshield of the target vehicle is not obstructed, then the distracted driving behavior detection device determines that the driver is engaging in the target distracted driving behavior.

[0063] In one feasible implementation, (1) if the driver has a special behavior of sudden illness, the distracted driving behavior detection device generates vehicle location information, sudden illness information and vehicle control information of the target vehicle where the driver is located, and the vehicle control information is used to control the target vehicle to stop and turn on the corresponding vehicle lights; (2) the distracted driving behavior detection device sends the vehicle location information and sudden illness information to the preset rescue platform.

[0064] For example, if a driver exhibits unusual behavior due to a sudden illness, the distracted driving detection device generates information about the vehicle's location, the sudden illness, and vehicle control. The sudden illness information indicates the driver's current condition, while the vehicle control information controls the target vehicle to stop and activate corresponding lights, such as hazard lights. The distracted driving detection device then sends the vehicle location and sudden illness information to a pre-set emergency response platform. This allows rescue personnel to locate the target vehicle based on its location, provide assistance to the driver, and control the vehicle to stop and activate the corresponding lights, thus preventing the vehicle from going out of control and causing a traffic accident. Simultaneously, it can also attract the attention of other drivers, facilitating timely assistance to drivers experiencing sudden illnesses.

[0065] In one feasible implementation, (1) if the length is less than a preset length, the distracted driving behavior detection device fills the initial driving video data to the preset length based on video interpolation to obtain the second candidate driving video data; (2) the distracted driving behavior detection device performs data enhancement processing on the second candidate driving video data to generate video data of the behavior to be identified. The data enhancement processing includes mirror flipping, video reversal, video cropping and / or video splicing.

[0066] For example, if the preset length is three minutes, and it is less than three minutes, the distracted driving behavior detection device will fill the initial driving video data to three minutes based on video interpolation to obtain the second candidate driving video data. The distracted driving behavior detection device will perform data augmentation processing on the second candidate driving video data to generate video data of the behavior to be identified. The data augmentation processing includes mirror flipping, video reversal, video cropping and / or video splicing.

[0067] In one feasible implementation, (1) if the driver has candidate distracted driving behavior, the distracted driving behavior detection device detects the driver’s special behavior and identifies the passenger behavior inside the target vehicle; (2) if the driver does not have special behavior and the passengers inside the target vehicle do not have any behavior that interferes with the driver’s safe driving, the distracted driving behavior detection device determines that the driver has target distracted driving behavior.

[0068] It should be noted that in this embodiment, the behavior of passengers inside the target vehicle causes the driver to perform special behaviors. This embodiment excludes scenarios where passengers interfere with the driver's normal driving and the driver makes suspected distracted driving behaviors. For example, if a passenger pulls the driver's hand, causing the driver to hold the steering wheel with one hand or take both hands off the steering wheel, the distracted driving behavior model will misjudge the passenger pulling the driver's behavior as distracted driving behavior.

[0069] For example, if a driver exhibits a candidate distracted driving behavior of taking both hands off the steering wheel, the distracted driving behavior detection device will perform special behavior detection on the driver and passenger behavior identification inside the target vehicle. If the driver does not exhibit any special behavior of taking both hands off the steering wheel due to passenger interference, and the passengers inside the target vehicle do not exhibit any behavior that interferes with the driver's safe driving, then the distracted driving behavior detection device will determine that the driver exhibits the target distracted driving behavior.

[0070] In this embodiment of the invention, a pre-set distracted driving behavior model is used to identify the video data of the driver's behavior to be identified. If the driver does not exhibit any special behavior and the target vehicle is in a normal driving condition, it is determined that the driver exhibits the target distracted driving behavior. This reduces the computation time for behavior recognition, achieves lightweight distracted driving behavior recognition, and eliminates the possibility of misjudgment of distracted driving behavior caused by the driver's special behavior and the vehicle's driving condition, thereby improving the accuracy of distracted driving behavior recognition.

[0071] The above describes the method for detecting distracted driving behavior in embodiments of the present invention. The following describes the device for detecting distracted driving behavior in embodiments of the present invention. Please refer to [link / reference]. Figure 3 One embodiment of the distracted driving behavior detection device of the present invention includes:

[0072] The acquisition and processing module 301 is used to acquire the driver's initial driving video data and process the initial driving video data to obtain the video data of the behavior to be identified.

[0073] The recognition module 302 is invoked to call a preset distracted driving behavior model to recognize the video data of the behavior to be recognized. The feature mapping of the distracted driving behavior model is represented as A∈R N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time.

[0074] The detection module 303 is used to detect special behaviors of the driver and detect the driving status of the target vehicle where the driver is located if the driver has candidate distracted driving behaviors.

[0075] The determination module 304 is used to determine that the driver has engaged in distracted driving behavior if the driver does not exhibit any special behavior and the target vehicle is in normal driving condition.

[0076] In this embodiment of the invention, a pre-set distracted driving behavior model is used to identify the video data of the driver's behavior to be identified. If the driver does not exhibit any special behavior and the target vehicle is in a normal driving condition, it is determined that the driver exhibits the target distracted driving behavior. This reduces the computation time for behavior recognition, achieves lightweight distracted driving behavior recognition, and eliminates the possibility of misjudgment of distracted driving behavior caused by the driver's special behavior and the vehicle's driving condition, thereby improving the accuracy of distracted driving behavior recognition.

[0077] Please see Figure 4 Another embodiment of the distracted driving behavior detection device in this invention includes:

[0078] The acquisition and processing module 301 is used to acquire the driver's initial driving video data and process the initial driving video data to obtain the video data of the behavior to be identified.

[0079] The recognition module 302 is invoked to call a preset distracted driving behavior model to recognize the video data of the behavior to be recognized. The feature mapping of the distracted driving behavior model is represented as A∈R N*C*T*H*W Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time.

[0080] The detection module 303 is used to detect special behaviors of the driver and detect the driving status of the target vehicle where the driver is located if the driver has candidate distracted driving behaviors.

[0081] The determination module 304 is used to determine that the driver has engaged in distracted driving behavior if the driver does not exhibit any special behavior and the target vehicle is in normal driving condition.

[0082] Optionally, the acquisition and processing module 301 is specifically used for:

[0083] Acquire the driver's initial driving video data and determine whether the video length of the initial driving video data is greater than or equal to the preset length;

[0084] If the length is greater than or equal to the preset length, the initial driving video data is sampled based on the preset length to obtain the first candidate driving video data;

[0085] Data augmentation processing is performed on the first candidate driving video data to generate video data of the behavior to be identified. The data augmentation processing includes mirror flipping, video reversal, video cropping and / or video stitching.

[0086] Optional, the distracted driving behavior detection device also includes:

[0087] The acquisition and generation module 305 is used to acquire the driver's historical driving video data and train a preset model based on the historical driving video data to generate a distracted driving behavior model. The historical driving video data includes multiple distracted driving behaviors of the driver.

[0088] Optionally, the acquisition module 305 is specifically used for:

[0089] Acquire historical driving video data of drivers and classify multiple distracted driving behaviors in the historical driving video data to obtain video data of multiple distracted driving behavior types;

[0090] The preset model is set based on preset parameters to obtain the preset model after setting.

[0091] The preset model is inserted into the residual network and data augmentation is performed to generate candidate models.

[0092] Candidate models are trained using video data of multiple types of distracted driving behavior to generate distracted driving behavior models.

[0093] Optionally, the detection module 303 includes:

[0094] The feature map generation unit 3031 is used to generate a driver behavior feature map based on the video data of the behavior to be identified if the driver has candidate distracted driving behavior.

[0095] The determining unit 3032 is used to determine that the driver does not have any special behavior if the behavior feature map does not match the corresponding preset special behavior feature map.

[0096] The vehicle detection unit 3033 is used to detect the driving status of the target vehicle where the driver is located.

[0097] Optionally, the vehicle detection unit 3033 is specifically used for:

[0098] Acquire images of the road ahead and the windshield of the target vehicle where the driver is located;

[0099] Perform road detection on the image of the road ahead and identify obstructions on the image of the windshield;

[0100] If the target vehicle meets the preset conditions, it is determined that the target vehicle is in normal driving condition. The preset conditions are that the road ahead is open to traffic in the image of the road ahead and there are no obstructions on the windshield in the image of the windshield.

[0101] Optional, the distracted driving behavior detection device also includes:

[0102] The information generation module 306 is used to generate vehicle location information, sudden illness information and vehicle control information of the target vehicle where the driver is located if the driver has a special behavior of sudden illness. The vehicle control information is used to control the target vehicle to stop and turn on the corresponding vehicle lights.

[0103] The information sending module 307 is used to send vehicle location information and sudden illness information to a preset rescue platform.

[0104] In this embodiment of the invention, a pre-set distracted driving behavior model is used to identify the video data of the driver's behavior to be identified. If the driver does not exhibit any special behavior and the target vehicle is in a normal driving condition, it is determined that the driver exhibits the target distracted driving behavior. This reduces the computation time for behavior recognition, achieves lightweight distracted driving behavior recognition, and eliminates the possibility of misjudgment of distracted driving behavior caused by the driver's special behavior and the vehicle's driving condition, thereby improving the accuracy of distracted driving behavior recognition.

[0105] above Figure 3 and Figure 4 The distracted driving behavior detection device in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The distracted driving behavior detection device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0106] Figure 5 This is a schematic diagram of a distracted driving behavior detection device 500 provided in an embodiment of the present invention. The distracted driving behavior detection device 500 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 510 (e.g., one or more processors) and a memory 520, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 533 or data 532. The memory 520 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the distracted driving behavior detection device 500. Furthermore, the processor 510 may be configured to communicate with the storage media 530 and execute the series of instruction operations in the storage media 530 on the distracted driving behavior detection device 500.

[0107] The distracted driving behavior detection device 500 may also include one or more power supplies 540, one or more wired or wireless network interfaces 550, one or more input / output interfaces 560, and / or one or more operating systems 531, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 5 The illustrated structure of the distracted driving behavior detection device does not constitute a limitation on the detection device and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0108] The present invention also provides a device for detecting distracted driving behavior. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the distracted driving behavior detection method described in the above embodiments.

[0109] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the distracted driving behavior detection method.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting distracted driving behavior, characterized in that, The method for detecting distracted driving behavior includes: Acquire the driver's initial driving video data and process the initial driving video data to obtain the video data of the behavior to be identified; The pre-defined distracted driving behavior model is invoked to identify the video data of the behavior to be identified. The feature mapping of the distracted driving behavior model is represented as follows: Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time. If the driver exhibits candidate distracted driving behavior, then the driver will be subject to special behavior detection, and the vehicle driving status of the target vehicle in which the driver is located will be detected. The special behavior includes at least the following: the driver taking both hands off the steering wheel due to a sudden illness; the driver looking around due to obstruction of vision caused by an obstruction on the windshield of the target vehicle; and the driver drinking water or eating when the road in front of the target vehicle is closed to traffic. If the driver does not exhibit any unusual behavior and the target vehicle is in normal driving condition, then it is determined that the driver is engaging in distracted driving behavior.

2. The method for detecting distracted driving behavior according to claim 1, characterized in that, The process of acquiring initial driving video data of the driver and processing the initial driving video data to obtain video data of the behavior to be identified includes: Acquire the driver's initial driving video data and determine whether the video length of the initial driving video data is greater than or equal to a preset length; If the initial driving video data is greater than or equal to the preset length, then the initial driving video data is sampled based on the preset length to obtain the first candidate driving video data. The first candidate driving video data is subjected to data augmentation processing to generate video data of the behavior to be identified. The data augmentation processing includes mirror flipping, video reversal, video cropping and / or video stitching.

3. The method for detecting distracted driving behavior according to claim 1, characterized in that, Before acquiring the driver's initial driving video data and processing the initial driving video data to obtain the video data of the behavior to be identified, the method further includes: The driver's historical driving video data is acquired, and a preset model is trained based on the historical driving video data to generate a distracted driving behavior model. The historical driving video data includes multiple distracted driving behaviors of the driver.

4. The method for detecting distracted driving behavior according to claim 3, characterized in that, The step of acquiring the driver's historical driving video data and training a preset model based on the historical driving video data to generate a distracted driving behavior model includes: The driver's historical driving video data is obtained, and multiple distracted driving behaviors in the historical driving video data are classified to obtain video data of multiple distracted driving behavior types. The preset model is set based on preset parameters to obtain the preset model after setting. The preset model is inserted into the residual network and data augmentation is performed to generate candidate models. The candidate model is trained based on video data of the multiple types of distracted driving behavior to generate a distracted driving behavior model.

5. The method for detecting distracted driving behavior according to claim 1, characterized in that, If the driver exhibits candidate distracted driving behavior, then special behavior detection is performed on the driver, and vehicle driving status detection is performed on the target vehicle where the driver is located, including: If the driver exhibits candidate distracted driving behavior, a behavioral feature map of the driver is generated based on the video data of the behavior to be identified. If the behavior feature map does not match the corresponding preset special behavior feature map, it is determined that the driver does not have any special behavior; The vehicle's driving status is detected for the target vehicle where the driver is located.

6. The method for detecting distracted driving behavior according to claim 5, characterized in that, The detection of the vehicle's driving status on the target vehicle where the driver is located includes: Acquire images of the road ahead and the windshield of the target vehicle where the driver is located; Road detection is performed on the image of the road ahead, and obstruction identification is performed on the image of the windshield. If the target vehicle meets the preset conditions, it is determined that the target vehicle is in normal driving condition. The preset conditions are that the road ahead in the image of the road ahead is open to traffic and there are no obstructions on the windshield in the image of the windshield.

7. The method for detecting distracted driving behavior according to any one of claims 1-6, characterized in that, After detecting the driver's special behavior and the vehicle's driving status if the driver exhibits candidate distracted driving behavior, the method further includes: If the driver exhibits a special behavior indicating a sudden illness, then the vehicle location information, sudden illness information, and vehicle control information of the target vehicle where the driver is located are generated. The vehicle control information is used to control the target vehicle to stop and turn on the corresponding vehicle lights. The vehicle location information and the information about the sudden illness are sent to a pre-set rescue platform.

8. A device for detecting distracted driving behavior, characterized in that, The device for detecting distracted driving behavior includes: The acquisition and processing module is used to acquire the driver's initial driving video data and process the initial driving video data to obtain the behavior video data to be identified. The recognition module is invoked to call a preset distracted driving behavior model to identify the video data of the behavior to be identified. The feature mapping of the distracted driving behavior model is represented as follows: Where N represents the batch size, C is the number of channels, T is the time dimension, and H and W represent the spatial resolution. The distracted driving behavior model moves the channels ±1 in the time dimension, moving 1 / 4 of the total number of channels each time. The detection module is used to detect special behaviors of the driver if the driver has candidate distracted driving behavior, and to detect the driving status of the target vehicle where the driver is located. The special behaviors include at least the driver taking his hands off the steering wheel due to a sudden illness, the driver looking around due to the windshield of the target vehicle being obstructed, and the driver drinking water or eating when the road in front of the target vehicle is closed to traffic. The determination module is used to determine that the driver has engaged in distracted driving behavior if the driver does not exhibit any special behavior and the target vehicle is in normal driving condition.

9. A device for detecting distracted driving behavior, characterized in that, The device for detecting distracted driving behavior includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the distracted driving behavior detection device to perform the distracted driving behavior detection method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the method for detecting distracted driving behavior as described in any one of claims 1-7.

Images