Driver action analysis method and device, vehicle and storage medium

Abstract

The invention relates to the technical field of safe driving, in particular to a driver action analysis method and device, a vehicle and a storage medium, and the method comprises the steps: determining a plurality of key skeleton joint points corresponding to the current driving action of a driver under the condition that the vehicle is in a target driving working condition, determining a target risk skeleton joint point of the driver according to the target coordinate information of each key skeleton joint point and the coordinate information of the target standard action joint point; and analyzing a target spatial distance and a target relative angle between the target risk skeleton joint point and a preset target dangerous action joint point, so as to determine at least one dangerous driving action of the driver according to an analysis result. Therefore, the problems of inaccurate data acquisition, high complexity of a multi-sensor data fusion algorithm, reduction of real-time performance and accuracy of driver action analysis and the like due to the fact that in-vehicle sensors are easily influenced by environmental factors in related technologies are solved.

Description

Technical Field

[0001] This application relates to the technical field of safe driving, and particularly to a method, device, vehicle, and storage medium for driver action analysis. Background Art

[0002] In the field of driver action analysis, existing technologies have developed to a relatively mature stage, and various different methods have been formed to monitor and evaluate the behavior state of drivers.

[0003] In related technologies, these systems rely on sensors installed inside the vehicle, such as cameras, infrared sensors, and radar, etc., to capture the actions and environmental information of the driver. By fusing and processing these data, a more comprehensive understanding of the driver's state can be obtained, such as whether the driver is in a state of fatigue driving or distracted driving. In addition, with the progress of computer vision technology, human pose estimation has also become one of the research hotspots. This method can track the body movements of the driver in real time and identify behaviors that may affect safe driving, such as dozing off and turning to look at the mobile phone.

[0004] However, in related technologies, due to the fact that in-vehicle sensors are vulnerable to environmental factors such as light and occlusion, the data collection is inaccurate. At the same time, the multi-sensor data fusion algorithm has a high complexity, increasing the computing cost and complexity, thus reducing the real-time performance and accuracy of driver action analysis, which urgently needs to be solved. Summary of the Invention

[0005] This application provides a method, device, vehicle, and storage medium for driver action analysis to solve the problems in related technologies, such as inaccurate data collection due to in-vehicle sensors being vulnerable to environmental factors such as light and occlusion, and the high complexity of the multi-sensor data fusion algorithm, increasing the computing cost and complexity, thus reducing the real-time performance and accuracy of driver action analysis.

[0006] The first aspect embodiment of this application provides a method for driver action analysis, including the following steps: when the vehicle is in a target driving condition, determining multiple key skeletal joint points corresponding to the driver's current driving action; based on the multiple key skeletal joint points, determining the target coordinate information of each key skeletal joint point of the driver's current driving action, and determining the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points; analyzing the target spatial distance and target relative angle between the target risk skeletal joint points and the pre-set target dangerous action joint points to obtain an analysis result, so as to determine at least one dangerous driving action of the driver according to the analysis result.

[0007] Optionally, in an embodiment of the present application, the determining of multiple key skeletal joint points corresponding to the driver's current driving action includes: recognizing the driver's current driving action by using the actual driving image of the driver; extracting at least one of the head joint point, shoulder joint point, elbow joint point, wrist joint point, hip joint point, knee joint point, and ankle joint point corresponding to the current driving action.

[0008] Optionally, in an embodiment of the present application, the determining of the driver's target risk skeletal joint points according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points includes: calculating the relative distances and relative angles between the target key skeletal joint points under the target standard action based on a pre-constructed standard action model; determining the coordinate information of the target standard action joint points by using the relative distances and relative angles between the target key skeletal joint points, so as to determine the driver's target risk skeletal joint points according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

[0009] Optionally, in an embodiment of the present application, after determining at least one dangerous driving action of the driver according to the analysis result, it further includes: generating a target multi-dimensional safety evaluation index of the driver's current driving action by using the at least one dangerous driving action of the driver; determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index; matching the target safety level with the target safety reminder method of the vehicle, so as to give a safety reminder to the driver according to the target safety reminder method.

[0010] Optionally, in an embodiment of the present application, after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index, it further includes: determining the type of dangerous behavior according to the at least one dangerous driving action of the driver, and generating a driving correction suggestion for the driver according to the type of dangerous behavior; generating a driving evaluation report of the current driving action based on the type of dangerous behavior, the driving correction suggestion, and the target multi-dimensional safety evaluation index; sending the driving evaluation report to a preset terminal, and displaying the driving evaluation report on the preset terminal.

[0011] The second aspect of the present application provides a device for analyzing driver actions, including: a first determination module, configured to determine a plurality of key skeletal joint points corresponding to the driver's current driving action when the vehicle is in a target driving condition; a second determination module, configured to determine the target coordinate information of each key skeletal joint point of the driver's current driving action based on the plurality of key skeletal joint points, and determine the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points; an analysis module, configured to analyze the target spatial distance and target relative angle between the target risk skeletal joint points and the pre-set target dangerous action joint points to obtain an analysis result, so as to determine at least one dangerous driving action of the driver according to the analysis result.

[0012] Optionally, in an embodiment of the present application, the first determination module includes: an identification unit, configured to identify the driver's current driving action by using the actual driving image of the driver; an extraction unit, configured to extract at least one of the head joint point, shoulder joint point, elbow joint point, wrist joint point, hip joint point, knee joint point, and ankle joint point corresponding to the current driving action.

[0013] Optionally, in an embodiment of the present application, the second determination module includes: a calculation unit, configured to calculate the relative distance and relative angle between the target key skeletal joint points under the target standard action based on a pre-constructed standard action model; a determination unit, configured to determine the coordinate information of the target standard action joint points by using the relative distance and relative angle between the target key skeletal joint points, so as to determine the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

[0014] Optionally, in an embodiment of the present application, the device of the embodiment of the present application further includes: a first generation module, configured to generate a target multi-dimensional safety evaluation index of the driver's current driving action by using at least one dangerous driving action of the driver after determining at least one dangerous driving action of the driver according to the analysis result; a second generation module, configured to determine the target safety level of the current driving action according to the target multi-dimensional safety evaluation index after determining at least one dangerous driving action of the driver according to the analysis result; a reminder module, configured to match the target safety reminder method of the vehicle by using the target safety level after determining at least one dangerous driving action of the driver according to the analysis result, so as to perform a safety reminder on the driver according to the target safety reminder method.

[0015] Optionally, in an embodiment of the present application, the device of the embodiment of the present application further includes: a third determination module, configured to determine a type of dangerous behavior according to at least one dangerous driving action of the driver after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index, and generate a driving correction recommendation for the driver according to the type of dangerous behavior; a third generation module, configured to generate a driving evaluation report for the current driving action based on the type of dangerous behavior, the driving correction recommendation, and the target multi-dimensional safety evaluation index after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index; a sending module, configured to send the driving evaluation report to a preset terminal after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index, and display the driving evaluation report on the preset terminal.

[0016] An embodiment of the third aspect of the present application provides a vehicle, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the method for analyzing driver actions as described in the above embodiment.

[0017] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the program is executed by a processor, the method for analyzing driver actions as described above is implemented.

[0018] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, and when the computer program is executed, it is used to implement the method for analyzing driver actions as described above.

[0019] In the embodiment of the present application, when the vehicle is in a target driving condition, a plurality of key skeletal joint points corresponding to the current driving action of the driver can be determined, and the target risk skeletal joint points of the driver can be determined according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points. Then, the target spatial distance and target relative angle between the target risk skeletal joint points and the target dangerous action joint points are analyzed to determine at least one dangerous driving action of the driver according to the analysis result, effectively improving the real-time performance and accuracy of driver action analysis. Thus, the problems in the related art are solved, that is, due to the fact that in-vehicle sensors are easily affected by environmental factors such as light and occlusion, the data collection is inaccurate, and at the same time, the multi-sensor data fusion algorithm has a high complexity, increasing the calculation cost and complexity, thereby reducing the real-time performance and accuracy of driver action analysis.

[0020] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Brief Description of the Drawings

[0021] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where:

[0022] Figure 1 is a flowchart of a method for analyzing driver actions provided according to an embodiment of the present application;

[0023] Figure 2 is a schematic structural diagram of a device for analyzing driver actions provided according to an embodiment of the present application;

[0024] Figure 3 is a schematic structural diagram of a vehicle provided according to an embodiment of the present application. Detailed Description of the Embodiments

[0025] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application.

[0026] The method, device, vehicle, and storage medium for analyzing driver actions according to embodiments of the present application will be described below with reference to the accompanying drawings. In view of the problems in the related art mentioned in the above background art, that is, due to the fact that in-vehicle sensors are susceptible to environmental factors such as light and occlusion, resulting in inaccurate data collection, and at the same time, the multi-sensor data fusion algorithm has a high complexity, increasing the computing cost and complexity, thus reducing the real-time performance and accuracy of driver action analysis, the present application provides a method for analyzing driver actions. In this method, when the vehicle is in a target driving condition, multiple key skeletal joint points corresponding to the driver's current driving action can be determined, and the target risk skeletal joint points of the driver can be determined according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points. Then, the target spatial distance and target relative angle between the target risk skeletal joint points and the target dangerous action joint points are analyzed to determine at least one dangerous driving action of the driver according to the analysis results, effectively improving the real-time performance and accuracy of driver action analysis. Thus, the problems in the related art, that is, due to the fact that in-vehicle sensors are susceptible to environmental factors such as light and occlusion, resulting in inaccurate data collection, and at the same time, the multi-sensor data fusion algorithm has a high complexity, increasing the computing cost and complexity, thus reducing the real-time performance and accuracy of driver action analysis, are solved.

[0027] Specifically, Figure 1 is a schematic flowchart of a method for analyzing driver actions provided according to an embodiment of the present application.

[0028] As Figure 1 shown, the method for analyzing the driver's actions includes the following steps:

[0029] In step S101, when the vehicle is in the target driving condition, determine multiple key skeletal joint points corresponding to the driver's current driving action.

[0030] In the example of the present application, the target driving condition is the condition when the vehicle is driving.

[0031] It can be understood that in the embodiments of the present application, when the vehicle is in the target driving condition, multiple key skeletal joint points corresponding to the driver's current driving action can be determined. For example, when the vehicle speed is greater than 0 km / h, the embodiments of the present application can obtain the driver's image through an in-vehicle camera or other image acquisition devices, and based on the deep learning algorithm, capture the driver's actions in real time, and extract multiple key skeletal joint points corresponding to the current driving action, which can more accurately identify the actual operation behavior, thereby improving the judgment accuracy and response speed of dangerous driving behaviors.

[0032] Among them, in an embodiment of the present application, determining multiple key skeletal joint points corresponding to the driver's current driving action includes: using the driver's actual driving image to identify the driver's current driving action; extracting at least one of the head joint point, shoulder joint point, elbow joint point, wrist joint point, hip joint point, knee joint point, and ankle joint point corresponding to the current driving action.

[0033] In the actual execution process, the embodiments of the present application can use the real-time collected driver's actual driving image to identify the driver's current driving action. Then, in combination with human body skeletal key point detection technologies, such as the OpenPose pose estimation algorithm, the yolo series, etc., the head joint point, shoulder joint point, elbow joint point, wrist joint point, hip joint point, knee joint point, or ankle joint point of the driver can be captured in real time. At the same time, the movement trajectory of the corresponding joint point can also be obtained to form multi-dimensional driving action analysis data. Thus, the embodiments of the present application can analyze the changes of key joint points in combination with specific driving scenarios, which helps to achieve fine monitoring of the driver's actions.

[0034] Among them, before using the real-time collected driver's actual driving image to identify the driver's current driving action, the actual driving image needs to undergo image preprocessing, such as denoising, enhancing contrast, grayscale conversion, etc., to improve the accuracy of recognition.

[0035] In step S102, based on multiple key skeletal joint points, the target coordinate information of each key skeletal joint point of the driver's current driving action is determined, and the target risk skeletal joint points of the driver are determined according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

[0036] In the embodiment of the present application, the target risk skeletal joint points are the skeletal joint points with potential safety hazards.

[0037] It can be understood that the embodiment of the present application can determine the target coordinate information of each key skeletal joint point of the driver's current driving action based on multiple key skeletal joint points. For example, after successfully detecting the key skeletal joint points in the above steps, the embodiment of the present application will further extract the two-dimensional coordinate information of each key skeletal joint point. Among them, the two-dimensional coordinate information of each key skeletal joint point is in pixels and represents the position of the key skeletal joint point on the image plane. For example, the coordinate of the shoulder joint point may be (x1, y1), the coordinate of the elbow joint point may be (x2, y2), and so on.

[0038] Next, after the embodiment of the present application obtains the two-dimensional coordinate information of each key skeletal joint point, according to the coordinate information of the standard action joint points determined based on the standard action model and the target coordinate information of each key skeletal joint point in the following steps, by calculating the deviation degree between the coordinate of each key skeletal joint point and the coordinate of the standard action joint point, the degree of deviation of the current driving action from the standard can be quantified, so as to determine the risk skeletal joint points of the driver, improve the detection accuracy of dangerous driving behaviors, and can achieve real-time monitoring and rapid response, thereby effectively reducing traffic accidents caused by unsafe driving behaviors.

[0039] For example, the embodiment of the present application sets one or more safety thresholds for each key skeletal joint point to determine whether the joint point is in a dangerous state. For example, excessive forward tilting of the head may mean fatigue driving, and hands leaving the steering wheel may indicate distracted driving, etc.; if the coordinate of a certain or certain key skeletal joint points exceeds the preset safety threshold, it is marked as a target risk skeletal joint point, which means that the relevant driving behavior may have potential safety hazards.

[0040] Among them, in an embodiment of the present application, determining the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points includes: calculating the relative distances and relative angles between the target key skeletal joint points under the target standard action based on a pre-constructed standard action model; using the relative distances and relative angles between the target key skeletal joint points to determine the coordinate information of the target standard action joint points, so as to determine the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

[0041] In an embodiment of the present application, the embodiment of the present application can construct a standard action model based on the suggestions of professional fitness personnel, biomechanical principles or a large amount of experimental data; among them, the standard action model is a pre-defined action template that describes the relative position relationship that should exist between each key skeletal joint point when performing a specific driving action.

[0042] As a possible implementation manner, the embodiment of the present application can calculate the relative distances and angles between each key skeletal joint point according to the relative position relationship of the key skeletal joint points defined in the standard action model. These relative distances and angles constitute the key skeletal joint point coordinate feature vector of the standard action. For example, the distance between the shoulder and the elbow, the angle between the elbow and the wrist, etc. can be calculated, and these values are combined into a multi-dimensional feature vector. Furthermore, the embodiment of the present application can determine the risk skeletal joint points of the driver in the current driving action according to the target coordinate information of each key skeletal joint point and the coordinate information of the standard action joint points, so as to determine the driving behaviors of the driver with potential safety hazards, effectively reducing the computational complexity and improving the efficiency of driver action analysis.

[0043] It should be noted that since everyone's body shape, height and limb proportions are different, when calculating the key skeletal joint point coordinate features of the standard action, normalization processing needs to be performed, so as to eliminate the influence of individual differences on feature calculation and make the action evaluation between different users more fair and accurate;

[0044] In step S103, analyze the target spatial distance and target relative angle between the target risk skeletal joint points and the pre-set target dangerous action joint points to obtain an analysis result, so as to determine at least one dangerous driving action of the driver according to the analysis result.

[0045] In an embodiment of the present application, the target dangerous action joint point features are determined based on traffic regulations requirements, driving safety standards or a large amount of experimental data, and describe the abnormal position relationships or dynamic features that may appear between each joint point in dangerous driving behaviors.

[0046] It can be understood that the embodiments of the present application can analyze the target spatial distance and target relative angle between the risk skeletal joint points and the pre-set dangerous action joint points. For example, the Euclidean distance can be used to calculate the spatial distance between the key skeletal joint points corresponding to the driver's current driving action and the dangerous action joint points, and the angular measure can be used to compare the relative angles between the key skeletal joints, etc. These quantitative indicators can objectively reflect the similarity or difference degree between the driver's action and the dangerous action.

[0047] Secondly, in addition to the basic quantitative comparison, the embodiments of the present application can further analyze the dynamic characteristics of the driver's action, such as the movement trajectory, speed, and acceleration of the joint points, etc. By comparing these dynamic characteristics, the present application can more comprehensively understand the process of the driver performing the action and identify potential dangerous behaviors or non-standard operations. Thus, the embodiments of the present application can obtain the analysis results and evaluate the operation safety of the driver. If the difference between the driver's action and the dangerous action exceeds a certain range, it will be determined that there is a safety hazard in the driving action, that is, it is determined that there is a dangerous driving action in the current driving action, thereby improving the discrimination ability for complex driving actions, and can more accurately and quickly identify specific dangerous driving behaviors, enhancing the scientific nature and real-time nature of risk warning.

[0048] Therefore, the embodiments of the present application can identify the key skeletal joint points of the driver's current action under the driving condition, and compare their coordinates with the coordinates in the standard action model to determine the target risk skeletal joint points. By analyzing the spatial distance and relative angle between these risk joint points and the pre-set dangerous action joint points, it can be determined whether the driver has dangerous driving behaviors. That is to say, the present application can monitor the driver's actions in real time and accurately, and identify potential dangerous behaviors such as fatigue driving and distracted driving, not only improving the detection accuracy of dangerous driving behaviors, effectively reducing the risk of traffic accidents, and enhancing driving safety.

[0049] Optionally, in an embodiment of the present application, after determining at least one dangerous driving action of the driver according to the analysis result, it further includes: generating a target multi-dimensional safety evaluation index of the driver's current driving action by using at least one dangerous driving action of the driver; determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index; and matching the target safety level with the target safety reminder method of the vehicle to perform a safety reminder on the driver according to the target safety reminder method.

[0050] In some embodiments, first, according to the analysis results of the key skeletal joint point features in the above steps, the degree of difference between the driver's actions and the standard safe driving actions is determined. Here, the degree of difference can be a simple numerical score or a comprehensive evaluation index including multiple dimensions (such as sitting posture accuracy, hand position standardization, head posture stability, etc.). Then, according to the preset threshold and determination rules, the operation safety of the driver is classified or graded. For example, the quality of driving actions can be divided into four levels: "safe", "low risk", "medium risk", and "high risk", or the dangerous behaviors can be divided into three categories: "minor violation", "medium violation", and "serious violation", which helps the driver more intuitively understand their own operation situation and clarify the direction for improvement. In addition, this application can also remind the driver through the dashboard indicator light when it is "low risk", issue a safety reminder through voice when it is "medium risk", and vibrate the steering wheel while issuing a safety reminder through voice when it is "high risk", so as to remind the driver of safety through different safety reminder methods, effectively improving the safety and reliability of driving and riding.

[0051] Among them, during the determination process, the embodiments of this application will also consider the driver's personal situation and historical data. For example, different determination criteria and thresholds can be adopted for novice drivers and experienced drivers, which can be specifically set by relevant technical personnel in this field and will not be specifically limited here. In addition, if the driver has had similar operation records before, the historical data can also be compared to evaluate the driver's behavior changes and driving habits, effectively improving the user's personalized experience.

[0052] Optionally, in an embodiment of this application, after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index, it further includes: determining the type of dangerous behavior according to at least one dangerous driving action of the driver, and generating a driving correction suggestion for the driver according to the type of dangerous behavior; generating a driving evaluation report for the current driving action based on the type of dangerous behavior, the driving correction suggestion, and the target multi-dimensional safety evaluation index; sending the driving evaluation report to a preset terminal, and displaying the driving evaluation report on the preset terminal.

[0053] In some embodiments, first, a set of error types and corresponding determination rules need to be predefined. These error types, that is, the types of dangerous behaviors, can be set based on common driving errors, traffic law requirements, or professional driving safety suggestions. For example, common error types may include incorrect sitting posture, improper hand position, abnormal head posture (such as fatigue driving), etc.

[0054] Next, in terms of the determination rules, specific determination conditions and thresholds can be set according to the characteristics of each error type. For example, for the error type of incorrect sitting posture, it will check whether the positions of the key joint points (such as the waist and shoulders) deviate from the expected positions in the standard driving posture model; for the error type of improper hand position, it will calculate whether the relative distance between the hand joint points and the steering wheel exceeds the safe range; for the error type of abnormal head posture, it will detect whether the tilt angle of the head joint points exceeds the set fatigue driving threshold. When it is detected that there are potential safety hazards in the driver's operations, they will be classified into the corresponding error types according to the preset determination rules. Thus, the driver can obtain specific feedback on their driving behavior and know which operations need to be improved and how to improve them.

[0055] It should be noted that since the height, body type and driving habits of each driver are different, individual differences need to be considered when classifying the actions that affect driving safety. To avoid misjudgment or overcorrection, flexible determination rules can be adopted and combined with the driver's personal data and historical records for comprehensive evaluation.

[0056] Secondly, if the difference between the driver's action and the dangerous action exceeds a certain range, it will be determined that there are potential safety hazards in this action, and corresponding warnings and driving correction suggestions will be given. Among them, the warnings and suggestions can include descriptions of dangerous behaviors, guidance on correction methods, and suggestions for safe driving, etc. Thus, a driving evaluation report of the current driving action can be generated based on the dangerous behavior type, driving correction suggestions and multi-dimensional safety evaluation indicators. The driving evaluation report can include the driver's operation safety score, dangerous behavior type and description, correction suggestions, and safe driving guidance, etc. The driving evaluation report will be sent to the driver's mobile phone and the vehicle central control screen, and the driving evaluation report will be displayed when the vehicle stops, effectively improving the safety of the user's driving and riding.

[0057] According to the method for analyzing the driver's actions proposed in the embodiment of the present application, when the vehicle is in the target driving condition, multiple key skeletal joint points corresponding to the driver's current driving action can be determined, and the target risk skeletal joint points of the driver can be determined according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points. Then, the target spatial distance and target relative angle between the target risk skeletal joint points and the target dangerous action joint points are analyzed to determine at least one dangerous driving action of the driver according to the analysis results, effectively improving the real-time performance and accuracy of the driver's action analysis. Thus, the problems in the related technology are solved, such as the inaccurate data collection due to the influence of environmental factors such as light and occlusion on the in-vehicle sensors, and the increase in calculation cost and complexity, which reduce the real-time performance and accuracy of the driver's action analysis.

[0058] Next, a device for analyzing driver actions according to an embodiment of the present application will be described with reference to the accompanying drawings.

[0059] Figure 2 It is a block diagram of a device for analyzing driver actions according to an embodiment of the present application.

[0060] As Figure 2 shown, the device 10 for analyzing driver actions includes: a first determination module 100, a second determination module 200, and an analysis module 300.

[0061] Specifically, the first determination module 100 is configured to determine a plurality of key skeletal joint points corresponding to the driver's current driving action when the vehicle is in a target driving condition.

[0062] The second determination module 200 is configured to determine the target coordinate information of each key skeletal joint point of the driver's current driving action based on the plurality of key skeletal joint points, and determine the driver's target risk skeletal joint points according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

[0063] The analysis module 300 is configured to analyze the target spatial distance and target relative angle between the target risk skeletal joint points and the pre-set target dangerous action joint points to obtain an analysis result, so as to determine at least one dangerous driving action of the driver according to the analysis result.

[0064] Optionally, in an embodiment of the present application, the first determination module 100 includes: an identification unit and an extraction unit.

[0065] Among them, the identification unit is configured to identify the driver's current driving action by using the driver's actual driving image.

[0066] The extraction unit is configured to extract at least one of the head joint point, shoulder joint point, elbow joint point, wrist joint point, hip joint point, knee joint point, and ankle joint point corresponding to the current driving action.

[0067] Optionally, in an embodiment of the present application, the second determination module 200 includes: a calculation unit and a determination unit.

[0068] Among them, the calculation unit is configured to calculate the relative distance and relative angle between the target key skeletal joint points under the target standard action based on a pre-constructed standard action model.

[0069] The determination unit is configured to determine the coordinate information of the target standard action joint points by using the relative distance and relative angle between the target key skeletal joint points, so as to determine the driver's target risk skeletal joint points according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

[0070] Optionally, in an embodiment of the present application, the device 10 of the embodiment of the present application further includes: a first generation module, a second generation module, and a reminder module.

[0071] Wherein, the first generation module is configured to generate a target multi-dimensional safety evaluation index of the driver's current driving action by using at least one dangerous driving action of the driver after determining at least one dangerous driving action of the driver according to the analysis result.

[0072] The second generation module is configured to determine the target safety level of the current driving action according to the target multi-dimensional safety evaluation index after determining at least one dangerous driving action of the driver according to the analysis result.

[0073] The reminder module is configured to match the target safety reminder method of the vehicle by using the target safety level after determining at least one dangerous driving action of the driver according to the analysis result, so as to give a safety reminder to the driver according to the target safety reminder method.

[0074] Optionally, in an embodiment of the present application, the device 10 of the embodiment of the present application further includes: a third determination module, a third generation module, and a sending module.

[0075] Wherein, the third determination module is configured to determine the type of dangerous behavior according to at least one dangerous driving action of the driver and generate a driving correction suggestion for the driver according to the type of dangerous behavior after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index.

[0076] The third generation module is configured to generate a driving evaluation report of the current driving action based on the type of dangerous behavior, the driving correction suggestion, and the target multi-dimensional safety evaluation index after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index.

[0077] The sending module is configured to send the driving evaluation report to a preset terminal and display the driving evaluation report on the preset terminal after determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index.

[0078] It should be noted that the foregoing explanation of the method embodiment for analyzing the driver's actions is also applicable to the device for analyzing the driver's actions in this embodiment, and will not be elaborated here.

[0079] The device for analyzing driver actions proposed according to the embodiments of the present application can, when the vehicle is in a target driving condition, determine multiple key skeletal joint points corresponding to the driver's current driving actions, and determine the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points. Then, analyze the target spatial distance and target relative angle between the target risk skeletal joint points and the target dangerous action joint points to determine at least one dangerous driving action of the driver according to the analysis results, effectively improving the real-time performance and accuracy of driver action analysis. Thus, it solves the problems in the related art that due to the fact that in-vehicle sensors are easily affected by environmental factors such as light and occlusion, the data collection is inaccurate, and it increases the computing cost and complexity, reducing the real-time performance and precision of driver action analysis, etc.

[0080] Figure 3 The structural schematic diagram of the vehicle provided by the embodiments of the present application. The vehicle may include:

[0081] A memory 301, a processor 302, and a computer program stored on the memory 301 and executable on the processor 302.

[0082] When the processor 302 executes the program, it implements the method for analyzing driver actions provided in the above embodiments.

[0083] Furthermore, the vehicle further includes:

[0084] A communication interface 303 for communication between the memory 301 and the processor 302.

[0085] The memory 301 is used to store a computer program executable on the processor 302.

[0086] The memory 301 may include a high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0087] If the memory 301, the processor 302, and the communication interface 303 are implemented independently, the communication interface 303, the memory 301, and the processor 302 can be interconnected via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 only a thick line is used to represent it in Figure 3 , but it does not mean that there is only one bus or one type of bus.

[0088] Optionally, in a specific implementation, if the memory 301, the processor 302, and the communication interface 303 are integrated on a single chip, the memory 301, the processor 302, and the communication interface 303 can communicate with each other through an internal interface.

[0089] The processor 302 may be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0090] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method for analyzing driver actions as described above is implemented.

[0091] This embodiment also provides a computer program product, including a computer program. When the computer program is executed, it is used to implement the method for analyzing driver actions as described above.

[0092] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc., mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0093] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0094] Any process or method description shown in the flowchart or described in other ways herein can be understood as representing a module, segment, or part of code including one or N executable instructions for implementing a customized logic function or process, and the scope of the preferred embodiments of this application includes additional implementations, where the functions can be executed in a manner that may not be in the order shown or discussed, including in a substantially simultaneous manner according to the functions involved or in the reverse order, which should be understood by those skilled in the art to which the embodiments of this application belong.

[0095] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a definitional sequence list of executable instructions for implementing logical functions, and can be embodied specifically in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection portion (electronic device) having one or N wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then storing it in a computer memory.

[0096] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and the like.

[0097] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the methods of the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0098] In addition, each functional unit in various embodiments of the present application may be integrated into one processing module, may exist separately as individual physical units, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0099] The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A method for analyzing driver motion, characterized in that: The following steps are involved: When the vehicle is in a target driving condition, determining a plurality of key skeletal joint points corresponding to the driver's current driving action; Based on the multiple key skeletal joint points, determine the target coordinate information of each key skeletal joint point of the driver's current driving action, and determine the driver's target risk skeletal joint point according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint point; The target spatial distance and the target relative angle between the target risk skeletal joint point and the preset target dangerous action joint point are analyzed to obtain an analysis result, so as to determine at least one dangerous driving action of the driver according to the analysis result.

2. The method according to claim 1, characterized in that The determining of a plurality of key skeletal joint points corresponding to the driver's current driving action includes: recognizing the current driving action of the driver by using the actual driving image of the driver; At least one of the head joint points, shoulder joint points, elbow joint points, wrist joint points, hip joint points, knee joint points and ankle joint points corresponding to the current driving action is extracted.

3. The method according to claim 1, characterized in that The step of determining the target risk skeletal joint point of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint point comprises: Based on a pre-built standard motion model, the relative distance and relative angle between target key skeletal joint points under the target standard motion are calculated; The relative distance and relative angle between the target key skeletal joint points are used to determine the coordinate information of the target standard action joint points, so as to determine the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

4. The method according to claim 1, characterized in that: After determining at least one dangerous driving action of the driver according to the analysis result, the method further includes: generating a target multi-dimensional safety evaluation index of the driver's current driving action using at least one dangerous driving action of the driver; Determining a target safety level of the current driving action according to the target multi-dimensional safety evaluation index; The target safety level is used to match a target safety reminder method of the vehicle, so as to provide a safety reminder to the driver according to the target safety reminder method.

5. The method according to claim 4, characterized in that After determining the target safety level of the current driving action according to the target multi-dimensional safety evaluation index, the method further includes: Determining a dangerous behavior type according to at least one dangerous driving action of the driver, and generating driving correction suggestions for the driver according to the dangerous behavior type; Generate a driving assessment report of the current driving action based on the risk behavior type, the driving correction suggestion and the target multi-dimensional safety evaluation index; The driving evaluation report is sent to a preset terminal, and the driving evaluation report is displayed on the preset terminal.

6. A device for analyzing driver movements, characterized in that: include: A first determination module is used to determine a plurality of key skeletal joint points corresponding to a current driving action of the driver when the vehicle is in a target driving condition; A second determination module is used to determine the target coordinate information of each key skeletal joint point of the driver's current driving action based on the multiple key skeletal joint points, and determine the driver's target risk skeletal joint point according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint point; The analysis module is used to analyze the target spatial distance and target relative angle between the target risk skeletal joint point and the preset target dangerous action joint point to obtain an analysis result, so as to determine at least one dangerous driving action of the driver according to the analysis result.

7. The device according to claim 6, characterized in that The first determining module comprises: an identification unit, configured to identify the current driving action of the driver using the actual driving image of the driver; An extraction unit is used to extract at least one of the head joint points, shoulder joint points, elbow joint points, wrist joint points, hip joint points, knee joint points and ankle joint points corresponding to the current driving action.

8. The device according to claim 6, characterized in that The second determining module comprises: A calculation unit, used for calculating the relative distance and relative angle between target key skeletal joint points under the target standard action based on a pre-built standard action model; A determination unit is used to determine the coordinate information of the target standard action joint points by using the relative distance and relative angle between the target key skeletal joint points, so as to determine the target risk skeletal joint points of the driver according to the target coordinate information of each key skeletal joint point and the coordinate information of the target standard action joint points.

9. A vehicle, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for analyzing driver motion as described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method for analyzing driver behavior as described in any one of claims 1 to 5.

Images