A method and system for identifying dangerous driving behavior based on action behavior characteristics

By combining ResNet-50 and Hourglass networks as the backbone of CenterNet and integrating feature weights, the accuracy and complex environmental adaptability of driver hand behavior analysis and dangerous driving behavior recognition in the prior art are solved, and more efficient identification is achieved and classification effects.

CN113971797BActive Publication Date: 2025-02-07SHANDONG HAILIANG INFORMATION TECH RES INST +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111473235.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-02
Publication Date
2025-02-07
Estimated Expiration
2041-12-02

AI Technical Summary

Technical Problem

The prior art faces problems of incomplete data collection, inaccurate target detection and difficult classification in driver hand behavior analysis and dangerous driving behavior identification, especially in complex driving environments.

Method used

A dangerous driving behavior recognition method based on action behavior characteristics is adopted. By combining ResNet-50 and Hourglass networks as the backbone of CenterNet, feature weights are fused to achieve more accurate dangerous driving behavior recognition.

Benefits of technology

It realizes more accurate driver hand behavior analysis and dangerous driving behavior recognition in complex driving environments, and improves the effect of identification and classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113971797B_ABST
    Figure CN113971797B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of computer vision technology, and provides a method and system for identifying dangerous driving behaviors based on action behavior characteristics. The network uses CenterNet as the target detection subject, and improves and combines ResNet‑50 and Hourglass networks to serve as the backbone of CenterNet. In order to improve accuracy and speed, improvements have also been made in multiple other modules, and a data set of dangerous driving behaviors of drivers under natural conditions has been collected to evaluate the network. The experimental part has conducted ablation experiments and comparative experiments, respectively, proving that the improvement of each module can have a positive impact on the performance of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a method and system for identifying dangerous driving behaviors based on action behavior characteristics. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] The large-scale development of the automobile industry has enhanced automobile technology, making it possible to basically ensure safety in terms of hardware, but there are still a large number of traffic accidents every day, of which 25% of collision accidents are caused by the driver's lack of concentration, such as distraction, fatigue, etc. Among them, dangerous driving behavior accounts for a large proportion, so it is necessary to monitor the driver's driving behavior.

[0004] Dangerous driving behavior detection focuses on the analysis of the driver's hand behavior during driving. It has many applications in the fields of human-computer interaction, human behavior understanding and motion recognition, and has attracted the attention of many researchers. The challenge is that, on the one hand, the hands cannot have complex postures and obvious features like the body to facilitate the identification of different behaviors, and the collected hand data often have problems such as external occlusion; on the other hand, in the driving environment, there are problems such as large changes in background and light intensity, and image jitter caused by car movement, which make it difficult to detect and analyze the driver's hands. It can be seen that collecting more comprehensive data, accurate driver hand target detection and correct dangerous behavior classification are all crucial in this study.

[0005] In terms of using the driver's hands to analyze dangerous driving behaviors, the research mainly focuses on traditional algorithm-based, device-based, and deep learning-based research. Research based on traditional algorithms is widely used and can take into account multi-level and wide-field recognition and classification, but the amount of calculation is huge and it is difficult to achieve accurate recognition results. The device-based method can accurately distinguish the subtle differences between different actions, and it is fast and has a wide range of application scenarios. However, since it is an invasive device, it will have a certain impact on the user's normal behavior during actual use, so this method is basically not considered in a driving environment that needs to ensure safety. Summary of the invention

[0006] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a method and system for identifying dangerous driving behaviors based on action behavior characteristics. The system comprehensively considers the driver's hands, the objects in his hands and the corresponding actions as features and inputs them into the target detection network for dangerous driving behavior identification, thereby realizing the judgment of whether the driver is in a dangerous driving state. The key technology is that ResNet-50 and Hourglass are processed and combined separately, and used together as the backbone of CenterNet.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A first aspect of the present invention provides a method for identifying dangerous driving behaviors based on action behavior characteristics, comprising the following steps:

[0009] Obtaining videos of natural dangerous driving behaviors to be identified during the driver's driving process;

[0010] Preprocessing the video of the natural dangerous driving behavior to be identified to obtain an image of the natural dangerous driving behavior to be identified;

[0011] The preprocessed natural dangerous driving behavior image to be identified is input into the trained dangerous driving behavior detection model to obtain the driver's dangerous driving behavior identification result; wherein, the dangerous driving behavior detection model uses CenterNet as the backbone network, and the backbone network of CenterNet includes Hourglass network and ResNet-50 network, each network channel outputs the corresponding original feature map, and the original feature map is processed separately to obtain the feature weight of each network channel, and the feature weights of different network channels are then applied to the network channel corresponding to the original feature map, and the original features are recalibrated and fused.

[0012] A second aspect of the present invention provides a dangerous driving behavior identification system based on action behavior characteristics, comprising:

[0013] The data acquisition module is configured to: acquire a video of a natural dangerous driving behavior to be identified during the driver's driving process;

[0014] The data preprocessing module is configured to: preprocess the video of the natural dangerous driving behavior to be identified to obtain an image of the natural dangerous driving behavior to be identified;

[0015] The dangerous driving behavior recognition module is configured to: input the preprocessed natural dangerous driving behavior image to be identified into the trained dangerous driving behavior detection model to obtain the driver's dangerous driving behavior recognition result; wherein the dangerous driving behavior detection model uses CenterNet as the backbone network, and the backbone network of CenterNet includes the Hourglass network and the ResNet-50 network. Each network channel outputs the corresponding original feature map, and the original feature map is processed separately to obtain the feature weight of each network channel, and the feature weights of different network channels are then applied to the network channel corresponding to the original feature map, and the original features are recalibrated and fused.

[0016] A third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the above-mentioned method for identifying dangerous driving behavior based on action behavior characteristics.

[0018] A fourth aspect of the present invention provides a computer device.

[0019] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the above-mentioned method for identifying dangerous driving behaviors based on action behavior characteristics are implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention obtains the feature weight of each network channel by processing the original feature map respectively, applies the feature weights of different network channels to the network channels corresponding to the original feature map, recalibrates the original features and then fuses them, so as to combine good features and remove bad features, thereby achieving better recognition and classification effects.

[0022] The present invention processes ResNet-50 and Hourglass separately and combines the feature maps to use them together as the backbone of CenterNet. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0024] Figure 1 It is a schematic diagram of the dangerous driving behavior recognition network structure;

[0025] Figure 2(a)-Figure 2(b)It is the location of the dashcam that collects data;

[0026] Figure 3(a)-Figure 3(j) This is an example of dangerous driving behavior data of drivers;

[0027] Figure 4(a)-Figure 4(b) It is the network performance of HAR-Net;

[0028] Figure 5(a)-Figure 5(e) It is the AP performance of each category in HAR-Net;

[0029] Figure 6(a)-Figure 6(f) This is the result of HAR-Net dangerous driving behavior recognition. DETAILED DESCRIPTION

[0030] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0031] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0032] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0033] Embodiment 1

[0034] like Figure 1 As shown, this embodiment provides a method for identifying dangerous driving behaviors based on action behavior characteristics, including:

[0035] S1: Obtain the natural dangerous driving behavior image to be identified;

[0036] like Figure 2(a)-Figure 2(b) As shown, a driving recorder is installed on the upper right side of the steering wheel and the right side of the co-pilot glass, respectively, and the driving recorder is used to collect driving behavior images;

[0037] During the experimental phase, driving behavior videos of 14 drivers were collected, including 5 girls and 9 boys, aged between 17 and 22 years old, with driving experience ranging from 1 to 5 years.

[0038] The driver's actions include making phone calls, smoking, playing with mobile phones, drinking water and eating. Among them, the calling actions include making phone calls with the left hand and making phone calls with the right hand; the smoking actions include smoking with the left hand, smoking with the right hand, steering the steering wheel with the left hand holding a cigarette, steering the steering wheel with the right hand holding a cigarette, flicking ash from the window with the left hand and flicking ash from the window with the right hand; the playing mobile phone actions include holding the mobile phone with the left hand and holding the mobile phone with the right hand; the drinking water actions include drinking water with the left hand, drinking water with the right hand, steering the steering wheel with a water bottle held in the left hand and steering the steering wheel with a water bottle held in the right hand; the eating actions use relatively obvious food collection, and the main actions include eating with the left hand, eating with the right hand, steering the steering wheel with food held in the left hand and steering the steering wheel with food held in the right hand.

[0039] The driving environment when collecting data was mainly during the day, including dark environments and complex high-light environments.

[0040] S2: preprocessing the natural dangerous driving behavior image to be identified;

[0041] like Figure 3(a)-Figure 3(j) As shown in the figure, the collected video is further screened, and then each frame of the video is converted into a picture and saved. The saved pictures are classified, and the same driving behaviors are grouped together. The LabelImg annotation tool is used to annotate the data, and the text format file is obtained as the true value of the data set.

[0042] During the experimental stage, the ratio of the training set to the test set was approximately 4:1, with 71,826 data sets in the training set and 17,919 data sets in the test set, and there was no overlap between the two parts of data.

[0043] S3: inputting the preprocessed natural dangerous driving behavior image to be identified into the trained dangerous driving behavior detection model to obtain the driver's dangerous driving behavior identification result;

[0044] The dangerous driving behavior detection model adopts CenterNet network, which includes Hourglass network and ResNet-50 network; each network channel outputs the corresponding original feature map, and the original feature map is processed to obtain the feature weight of each network channel. The feature weights of different network channels are applied to the network channel corresponding to the original feature map, and the original features are recalibrated and fused. By combining the extracted feature maps of the two networks, it is equivalent to combining good features and removing bad features, so as to achieve better recognition and classification effects. This network is named HAR-Net.

[0045] S301: The construction process of the Hourglass network includes: the Hourglass network uses dilated convolution and adds a DCM (Dilated Hourglass Modules) structure, and its design goal is to make full use of features at different levels and reduce information loss. Traditional methods tend to use up and down sampling to expand the perception domain and obtain higher-level features. However, this up and down sampling structure usually leads to the loss of information and resolution, which will have a great impact on determining the precise location of the target. Therefore, dilated convolution is used to construct the basic module.

[0046] A DCM consists of three layers, including two ordinary convolutional layers, a dilated convolutional layer added between the two ordinary convolutional layers, and a jump connection across the dilated convolutional layer for connecting the first ordinary convolutional layer and the third ordinary convolutional layer. The jump connection is used to propagate the feature information extracted from the first ordinary convolutional layer, and the feature map output by the entire DCM module obtained by the dilated convolutional layer and the jump connection is added pixel by pixel, and the generated feature map is input to the second ordinary convolutional layer.

[0047] In the experimental stage, the kernel size of these three layers is 3, and the dilation rate of the dilated convolution layer is 2. In the skip connection, 1×1 convolution is used to achieve cross-channel information interaction and integration without requiring too many convolution kernel parameters.

[0048] The advantage of the above scheme is that the residual module is replaced by DCM through the improved Hourglass network, thereby reducing the subsampling time and reducing information loss. Therefore, the dilated convolution is used to construct the basic module, which can expand the receptive field while keeping the resolution unchanged.

[0049] S302: Modify the convolution kernel size and step size of the downsampling layer of the ResNet-50 network, and add an average pooling layer;

[0050] In the existing ResNet-50 network structure, the downsampling part is divided into two paths, called path A and path B. Path A uses a 1×1 convolution kernel with a step size of 2, so it ignores three-quarters of the feature information. Similarly, the convolution layer in path B also ignores three-quarters of the feature information. Based on the above problems, the downsampling layer of the ResNet-50 network structure is improved by modifying the size and step size of the convolution kernel;

[0051] In path A, modify the stride of the two convolutional layers, set the stride of the 3×3 convolution kernel to 2, and set the stride of the remaining convolutional layers to 1.

[0052] Improvements have also been made to the B path by setting the stride of the convolutional layer to 1 and adding a 2×2 average pooling with a stride of 2 before the convolutional layer.

[0053] S303: Add the attention mechanism module SENet (Squeeze-and-Excitation-Network) to the ResNet-50 and Hourglass networks respectively. The core idea is to learn the feature weights according to the loss value, so that the effective feature map has a large weight, and the invalid or ineffective feature map has a small weight. In this way, the model is trained to achieve better results.

[0054] S304: Fusion of the output results of the Hourglass network and the ResNet-50 network;

[0055] The fusion method of the Hourglass network and the ResNet-50 network includes a Loss value combination method and a feature map combination method;

[0056] Method 1: In the Loss value combination, the corresponding Loss value is obtained and combined after extracting features through the ResNet-50 and Hourglass networks. The Loss value represents the distance between the model output and the actual result.

[0057] The combined formula is:

[0058] L=L1+L2 (1)

[0059] Among them, L is the Loss value after the entire CenterNet network is combined, and L1 and L2 are the Loss values ​​after passing through ResNet-50 and Hourglass respectively.

[0060] The method based on Loss value combination is the most basic combination method. Even if the data is extracted in parallel through ResNet-50 and Hourglass networks to obtain the corresponding Loss values ​​and combine them, it can produce the effect of 1+1>2.

[0061] Method 2: Feature map combination, such as Figure 1 As shown in the figure, this network is named HAR-Net.

[0062] The last layer of Hourglass outputs the first feature map, and the last layer of ResNet-50 outputs the second feature map;

[0063] Filling the second feature map according to the size of the first feature map;

[0064] The first feature map and the padded second feature map are fused, the fused feature map is used as the overall feature map of the network, and the overall feature map of the network is output for detection and recognition.

[0065] For example, the data size of our input model is 512×512, the size of feature map A output by the last layer of Hourglass is also 512×512, while the size of feature map B output by the last layer of ResNet-50 is only 16×16, so feature map B must be padded to 512×512 first;

[0066] ResNet-50 will lose a lot of information as the number of network layers increases, so the size of the feature map will decrease. In this case, in order to combine the feature maps of the two networks, the feature map output by the last layer of ResNet-50 must be filled first.

[0067] The advantage of the feature map combination method over the loss value combination method is that this method does not simply combine the results of the two networks, but combines the feature maps extracted from the two networks, which is equivalent to combining good features and removing bad features to achieve better recognition and classification effects.

[0068] In order to improve the processing speed, the present invention improves the final Loss function based on the combination of the above-mentioned feature maps, and proposes to use SoftMax Loss and Center Loss together, using SoftMax Loss to separate different categories, and then using Center Loss to compress the same category, and finally perform identification and classification.

[0069] Among them, the SoftMax Loss function includes SoftMax and Cross-Entropy Loss, and the formula is as follows:

[0070]

[0071] Among them, L is the loss, S j It is the j-th value of the SoftMax output vector S, which represents the probability that the sample belongs to the j-th category. The range of j is also 1 to the number of categories T. Therefore, y is a 1×T vector, and only the value at the position corresponding to the true label is 1, and the other T-1 values ​​are all 0.

[0072] Therefore, formula (2) can be expressed as:

[0073] L=-logS j (3)

[0074] The principle of Center Loss is to set several center points for classification so that the features of different categories are as close to their respective center points as possible, that is, to reduce the distance within the class and increase the distance between classes. The formula is as follows:

[0075]

[0076] Here, x refers to the feature, and C is the category center corresponding to each sample in each batch. It has the same dimension as feature x and will be updated as the model is trained. It is worth noting that when calculating the category center, only the images of a certain category in the current batch are used to obtain the update amount of the category center, that is, the change of each category center is calculated only using the features of the images belonging to this category.

[0077] In the model, SoftMax Loss and Center Loss are used together, and λ is the weight of loss:

[0078] L=L s +λL c (5)

[0079] During the experiment, the collected and labeled data was divided into a training set and a test set at a ratio of 4:1. The training set was input into the network for training, and the network categories were set to five: eating, drinking, making phone calls, playing with mobile phones, and smoking.

[0080] Set 20 epochs as pre-training and 20 epochs as formal training. After the training is completed, save the weight file and select the weight with the smallest Loss value to input into the test network to judge the driver's dangerous driving behavior.

[0081] On this basis, an ablation experiment was conducted on the network, which was divided into the original Hourglass network, the original ResNet-50 network, the improved Hourglass network, the improved ResNet-50 network, the network with only the attention mechanism added, and the combined HAR-Net. It was concluded that the improvements in each part played a positive role in the network. Figure 4(a)-Figure 4(b) This is the network performance of HAR-Net. Figure 5(a)-Figure 5(e) It is the AP performance of each category in HAR-Net. Figure 6(a)-Figure 6(f) This is the result of HAR-Net dangerous driving behavior recognition.

[0082] The present invention uses CenterNet as the target detection subject, improves ResNet-50 and Hourglass networks respectively and combines them as the backbone of CenterNet. In order to improve accuracy and speed, improvements have also been made to multiple other modules, and a dataset of dangerous driving behaviors under natural conditions has been collected to evaluate the network. The experimental part conducted ablation experiments and comparative experiments respectively, proving that the improvement of each module can have a positive impact on the performance of the model. Among them, HAR-Network achieved 98.8% mAP on our dataset.

[0083] Embodiment 2

[0084] This embodiment provides a dangerous driving behavior identification system based on action behavior characteristics, including:

[0085] The data acquisition module is configured to: acquire a video of a natural dangerous driving behavior to be identified during the driver's driving process;

[0086] The data preprocessing module is configured to: preprocess the video of the natural dangerous driving behavior to be identified to obtain an image of the natural dangerous driving behavior to be identified;

[0087] The dangerous driving behavior recognition module is configured to: input the preprocessed natural dangerous driving behavior image to be identified into the trained dangerous driving behavior detection model to obtain the driver's dangerous driving behavior recognition result; wherein the dangerous driving behavior detection model uses CenterNet as the backbone network, and the backbone network of CenterNet includes the Hourglass network and the ResNet-50 network. Each network channel outputs the corresponding original feature map, and the original feature map is processed separately to obtain the feature weight of each network channel, and the feature weights of different network channels are then applied to the network channel corresponding to the original feature map, and the original features are recalibrated and fused.

[0088] Embodiment 3

[0089] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the method for identifying dangerous driving behaviors based on action behavior characteristics as described above are implemented.

[0090] Embodiment 4

[0091] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for identifying dangerous driving behaviors based on action behavior characteristics as described above are implemented.

[0092] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0093] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0097] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dangerous driving behavior identification method based on action behavior characteristics, characterized in that: The steps include: Obtaining videos of natural dangerous driving behaviors to be identified during the driver's driving process; Preprocessing the video of the natural dangerous driving behavior to be identified to obtain an image of the natural dangerous driving behavior to be identified; The preprocessed natural dangerous driving behavior image to be identified is input into the trained dangerous driving behavior detection model to obtain the driver's dangerous driving behavior identification result; wherein the dangerous driving behavior detection model uses CenterNet as the backbone network, and the backbone network of CenterNet includes the Hourglass network and the ResNet-50 network. Each network channel outputs the corresponding original feature map, and the original feature map is processed to obtain the feature weight of each network channel. The feature weights of different network channels are then applied to the network channel corresponding to the original feature map, and the original features are recalibrated and fused; The Hourglass network adopts a dilated convolutional network, which includes two ordinary convolutional layers and one dilated convolutional layer. A jump connection is set between the two ordinary convolutional layers before and after the dilated convolutional layer. The jump connection is used to transfer the feature information extracted from the first ordinary convolutional layer to the dilated convolutional layer to obtain a feature map, and the feature map is input into the second ordinary convolutional layer. The convolution kernel size and step size of the downsampling layer of the ResNet-50 network are modified, and an average pooling layer is added; In the structure of the ResNet-50 network, the downsampling part is divided into two paths, called path A and path B. In path A, the stride of the two convolutional layers is modified, the stride of the convolution kernel 3×3 is set to 2, and the stride of the other convolutional layers is set to 1; in path B, the stride of the convolutional layer is set to 1, and a 2×2 average pooling with a stride of 2 is added before the convolutional layer.

2. The method for identifying dangerous driving behaviors based on action behavior characteristics as claimed in claim 1, characterized in that: An attention mechanism module SENet is added to the Hourglass network and the ResNet-50 network respectively. The attention mechanism module SENet learns the feature weight of each network channel according to the loss value, so that the weight of the effective feature map is increased and the weight of the invalid or small effect feature map is reduced.

3. The method for identifying dangerous driving behaviors based on action behavior characteristics as claimed in claim 1, characterized in that: The Loss value fusion method is used to recalibrate the original features and then fuse them; Alternatively, the original features are recalibrated and then fused using the feature map fusion method.

4. The method for identifying dangerous driving behaviors based on action behavior characteristics as claimed in claim 1, characterized in that: The preprocessing of the natural dangerous driving behavior video to be identified includes: The collected videos are further screened, and then each frame of the video is converted into an image and saved. The saved images are classified, and the same driving behaviors are grouped together. The LabelImg annotation tool is used to annotate the data, and a text-formatted file is obtained as the true value of the natural dangerous driving behavior video dataset.

5. The method for identifying dangerous driving behaviors based on action behavior characteristics as claimed in claim 1, characterized in that: The natural dangerous driving behavior image to be identified includes hands, objects held in the hands, and actions.

6. A dangerous driving behavior identification system based on action behavior characteristics, using the dangerous driving behavior identification method based on action behavior characteristics according to any one of claims 1 to 5, characterized in that: include: The data acquisition module is configured to: acquire a video of a natural dangerous driving behavior to be identified during the driver's driving process; The data preprocessing module is configured to: preprocess the video of the natural dangerous driving behavior to be identified to obtain an image of the natural dangerous driving behavior to be identified; The dangerous driving behavior recognition module is configured to: input the preprocessed natural dangerous driving behavior image to be identified into the trained dangerous driving behavior detection model to obtain the driver's dangerous driving behavior recognition result; wherein the dangerous driving behavior detection model uses CenterNet as the backbone network, and the backbone network of CenterNet includes the Hourglass network and the ResNet-50 network. Each network channel outputs the corresponding original feature map, and the original feature map is processed separately to obtain the feature weight of each network channel, and the feature weights of different network channels are then applied to the network channel corresponding to the original feature map, and the original features are recalibrated and fused.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a dangerous driving behavior identification method based on action behavior characteristics as described in any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the dangerous driving behavior identification method based on action behavior characteristics as described in any one of claims 1-5 are implemented.

Citation Information

Patent Citations

  • Driver behavior recognition method based on human body feature fine classification

    CN110309723A

  • Driver state monitoring method based on convolutional neural network

    CN110837815A