Non-driving behavior risk protection method and system based on driving posture recognition

Abstract

The invention relates to a non-driving behavior risk protection method and system based on driving posture recognition, and the method comprises the steps: receiving video data of a driver, carrying out the joint point recognition, and obtaining the driving posture of the driver at each moment; performing similarity comparison on the driving posture of the driver at each moment and a standard driving posture, and calculating the degree of dislocation, so as to perform safety scoring on the degree of dislocation; non-driving behavior duration is determined according to a posture similarity threshold value, and corresponding off-location time is determined according to safety measure reserved time, so that the off-location time is compared with human body reaction demand time to perform safety scoring on the off-location time; and comprehensively considering the safety scoring result of the off-location degree and the safety scoring result of the corresponding off-location time by adopting a matrix scoring mode, and determining a risk degree grade scoring result, thereby executing a corresponding safety protection strategy. Compared with the prior art, the man-machine interaction experience and efficiency of the intelligent driving vehicle can be greatly improved, false alarms are reduced, and the safety of the vehicle is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving, and in particular to a non-driving behavior risk protection method and system based on driving posture recognition. Background Art

[0002] With the increasing popularity of intelligent driving vehicles, during the driving process, drivers will engage in many non-driving related activities. The existing distraction behavior recognition systems in the market have the problem of frequent false alarms, and the human-computer interaction experience is very poor. Many consumers even directly turn off the prompts of the system, which poses a great potential safety hazard. In addition, the current market products cannot provide different safety protections for different non-driving behaviors, and there is little research on the risk level of non-driving behaviors.

[0003] The main reason is that the definition and recognition of the risk level require real-time and accurate recognition of the current driver's limb position, and quantitative research on the driving posture, involving real-time quantitative analysis of the deviation degree and deviation time of the human body joint points of the driving posture, which is very difficult. Summary of the Invention

[0004] The purpose of the present invention is to overcome the above-mentioned defects existing in the prior art, and provide a non-driving behavior risk protection method and system based on driving posture recognition, so as to realize the recognition of the non-driving behavior risk level based on driving posture recognition and the generation of safety protection strategies.

[0005] The purpose of the present invention can be achieved through the following technical solutions: A non-driving behavior risk protection method based on driving posture recognition includes the following steps: Receiving driver video data; Performing joint point recognition on the driver video data through a vision-based human posture recognition system to obtain the driving postures of the driver at each moment; Comparing the driving postures of the driver at each moment with a preset standard driving posture, calculating the out-of-position degree of each moment's driving posture, and performing a safety score on the out-of-position degree of non-driving behaviors according to the out-of-position degree; Comparing the similarity comparison result with a preset posture similarity threshold to determine the duration of non-driving behavior, and determining the corresponding out-of-position time according to the preset safety measure reservation time, comparing the out-of-position time with the human reaction requirement time, and performing a safety score on the out-of-position time of non-driving behaviors; Adopting a matrix scoring method to comprehensively consider the safety score result of the out-of-position degree and the safety score result of the corresponding out-of-position time to determine the non-driving behavior risk level scoring result, and thus execute the corresponding safety protection strategy.

[0006] Further, the degree of deviation is calculated based on the head deflection ratio and joint deviation ratio in the similarity comparison result, and the corresponding calculation expression is: Deviation= + In the formula, Deviation is the degree of deviation of the overall driving posture, is the calculation weight coefficient of the head deflection ratio accounting for the degree of deviation of the overall driving posture, is the head deflection ratio, is the calculation weight coefficient of each joint deviation ratio accounting for the degree of deviation of the overall driving posture, is the joint deviation ratio.

[0007] Further, the scoring process of the degree of deviation of the overall driving posture is specifically as follows: Pre-divide the deviation degree intervals corresponding to different safety levels, and set the scores corresponding to each interval; Obtain the corresponding scoring value according to the interval to which the calculated degree of deviation belongs.

[0008] Further, the calculation process of the deviation time is specifically as follows: Compare the similarity comparison result with the preset posture similarity threshold. If it exceeds the posture similarity threshold, start timing, determine the duration of the non-driving behavior, and calculate the corresponding deviation time in combination with the current driving scenario and the reserved time for safety measures. The corresponding calculation expression is: T= In the formula, T is the deviation time, is the duration of the non-driving behavior, is the scenario coefficient, is the reserved time for safety measures.

[0009] Further, the scoring process of the deviation time is specifically as follows: Pre-divide the deviation time intervals corresponding to different safety levels, and set the scores corresponding to each interval; Obtain the corresponding scoring value according to the deviation time interval to which the calculated deviation time belongs.

[0010] Further, the process of determining the scoring result of the non-driving behavior risk degree level by the matrix scoring method is specifically as follows: Multiply the safety scoring result of the degree of deviation and the safety scoring result of the deviation time to obtain the scoring result of the non-driving behavior risk degree level.

[0011] Further, the execution judgment process of the safety protection strategy includes: Previously, intervals corresponding to different risk levels are defined for the scoring results of the non-driving behavior risk level. The risk levels include high risk, relatively high risk, medium risk, and low risk; If it is a high risk, the corresponding countermeasure is vehicle takeover; If it is a relatively high risk, the corresponding countermeasure is to request vehicle takeover; If it is a medium risk, the corresponding countermeasure is to prompt the risk; If it is a low risk, the corresponding countermeasure is not to prompt.

[0012] Furthermore, the way of prompting the risk includes voice prompting through a horn and vibration prompting through a seat drive structure.

[0013] The present invention also provides a system for implementing a non-driving behavior risk protection method based on driving posture recognition as described above, including: A camera module for acquiring driver video data; A calculation module for executing a non-driving behavior risk protection method based on driving posture recognition as described above; An actuator for performing actions according to the safety protection strategy output by the calculation module.

[0014] Furthermore, the system further includes: A display module for displaying the data processing results of the calculation module.

[0015] Compared with the prior art, the present invention has the following advantages: (1) The present invention first determines the driving posture by identifying the joint points of the driver video data, compares it with the standard driving posture to calculate the degree of displacement, and thus on the one hand, performs a safety score for the degree of displacement of non-driving behavior according to the degree of displacement, and on the other hand, performs a safety score for the displacement time of non-driving behavior according to the displacement time; finally, a matrix scoring method is adopted to comprehensively consider the safety scoring results of the degree of displacement and the corresponding safety scoring results of the displacement time to determine the scoring result of the non-driving behavior risk level, and then execute the corresponding safety protection strategy; The present invention realizes the definition of the risk level of non-driving behavior, provides a feasible solution for hierarchical safety protection based on the risk level, can provide personalized safety protection strategies, can greatly improve the driver's human-computer interaction experience in the actual vehicle application, reduce false alarms, improve the safety protection effect, enable the driver to be better protected during the collision process, and greatly reduce the degree of personal injury. Description of the Drawings

[0016] Figure 1It is a schematic flowchart of a non-driving behavior risk protection method based on driving posture recognition provided in an embodiment of the present invention; Figure 2 It is a schematic structural diagram of a non-driving behavior risk protection system based on driving posture recognition provided in an embodiment of the present invention. Detailed implementation manners

[0017] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and illustrated herein generally may be arranged and designed in a variety of different configurations.

[0018] Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed present invention, but is merely representative of selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0019] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0020] Embodiment 1 As Figure 1 shown, this embodiment provides a non-driving behavior risk protection method based on driving posture recognition, including the following steps: S1: Receive driver video data; S2: Perform joint point recognition on the driver video data through a vision-based human posture recognition system to obtain the driving postures of the driver at each moment; S3: Compare the driving postures of the driver at each moment with a preset standard driving posture, calculate the displacement degree of each moment's driving posture, and perform a safety score on the displacement degree of non-driving behavior according to the displacement degree; S4: Compare the similarity comparison result with a preset posture similarity threshold to determine the duration of non-driving behavior, and determine the corresponding displacement time according to the reserved time of the preset safety measure. Compare the displacement time with the human reaction demand time to perform a safety score on the displacement time of non-driving behavior; S5: Adopt a matrix scoring method to comprehensively consider the safety score result of the displacement degree and the safety score result of the corresponding displacement time to determine the non-driving behavior risk degree level score result, and thus execute the corresponding safety protection strategy.

[0021] Specifically, this solution considers two dimensions of the departure degree and departure time of the driving posture, and uses the product of the fitting scores (ranging from 0 to 5 points) of the two dimensions as the risk degree level score, ranging from 0 to 25 points.

[0022] First, through a vision-based human posture recognition system, obtain the current person's driving posture; then, calculate the departure degree of the current driving posture through the similarity comparison algorithm between the current driving posture and the standard driving posture. The formula for the departure degree is as follows: Deviation= + In the formula, Deviation is the overall driving posture departure degree, is the calculation weight coefficient of the head deflection ratio accounting for the overall driving posture departure degree, is the head deflection ratio, is the calculation weight coefficient of the joint deviation ratio accounting for the overall driving posture departure degree, is the joint deviation ratio.

[0023] The specific process of scoring the overall driving posture departure degree is as follows: Pre-divide the departure degree intervals corresponding to different safety levels and set the scores corresponding to each interval; According to the interval to which the calculated departure degree belongs, obtain the corresponding score value.

[0024] In this embodiment, the departure degree score is fitted to a score of 0 to 5, as shown in Table 1.

[0025] Table 1 Overall driving posture departure degree scoring table By referring to the different reaction times of the human body in the literature and leaving a reaction time for AEB (Automatic Emergency Braking System), start timing from when it is determined that non-driving behavior is engaged until the end of the behavior.

[0026] The specific process of calculating the departure time is as follows: According to the comparison between the similarity comparison result and the preset posture similarity threshold, if it exceeds the posture similarity threshold, start timing, determine the duration of non-driving behavior, and combine the current driving scenario and safety measures to reserve time to calculate the corresponding departure time. The corresponding calculation expression is: T= In the formula, T is the effective departure time, is the duration of non-driving behavior, is the scenario coefficient, representing the weight coefficient of different scenarios, Reserve time for safety measures, such as the AEB intervention time, and temporarily take the constant value of 1.6 s.

[0027] The scoring process for the off-position time is specifically as follows: Pre-divide the off-position time intervals corresponding to different safety levels, and set the scores corresponding to each interval; According to the off-position time interval to which the calculated off-position time belongs, obtain the corresponding scoring value.

[0028] In this embodiment, different off-position times are fitted to scores from 0 to 5, as shown in Table 2 specifically.

[0029] Table 2 Scoring of off-position time for non-driving behaviors The off-position time risk level is divided according to the limit reaction time required by different parts of the human body. For example, it takes at least 0.3 s for the eyes to return from deviation to the straight-ahead position, and the shortest reaction time of the human nervous system is 0.5 s. When performing complex tasks, there will be cognitive distraction, and the total reaction of a person requires at least 3 s. Based on the above parameters, the time period of 0 - 0.3 s with a short off-position time and only requiring visual reaction is divided into a low-risk interval. If the off-position time continues to increase, 0.8 s (0.3 s plus 0.5 s), which requires both visual and neural extreme reactions, is used as the grading boundary, and 0.3 s - 0.8 s is used as the medium-risk interval. If the human body needs to perform complex tasks, there will be cognitive analysis at this time, and a longer reaction time is required. 0.8 - 3 s is used as the high-risk time interval. When the off-position time of the human body exceeds 3 s, when facing an emergency and the driver needs to take over driving, since the human body needs at least 3 s or even longer to effectively take over the vehicle, it is very likely that an accident will be unavoidable at this time. Therefore, the off-position time exceeding 3 s has an extremely high risk.

[0030] Through the matrix scoring method, the risk degree scores of different driving postures at the current moment can be obtained, and the score levels will change dynamically with time. For example, for an action with the same risk level, as the duration increases, its risk degree will continuously increase; while for a high-risk driving posture with a short duration, its risk degree score will not be very high.

[0031] The matrix scoring method can be: Multiply the safety scoring result of the off-position degree by the safety scoring result of the off-position time to obtain the risk degree level scoring result of non-driving behaviors, and obtain the risk degree R.

[0032] In this embodiment, the matrix risk degree scoring table is shown in Table 3.

[0033] Table 3 Matrix risk degree scoring The execution judgment process of the safety protection strategy includes: Previously, different intervals corresponding to different risk levels were divided for the scoring results of the non-driving behavior risk level. The risk levels include high risk, relatively high risk, medium risk, and low risk; If it is a high risk, the corresponding countermeasure is vehicle takeover; If it is a relatively high risk, the corresponding countermeasure is to request vehicle takeover; If it is a medium risk, the corresponding countermeasure is to prompt the risk; If it is a low risk, the corresponding countermeasure is not to prompt.

[0034] In this embodiment, different targeted safety protection strategies are defined, as shown in Table 4 specifically.

[0035] Table 4 Risky driving posture countermeasure strategies Embodiment 2 As Figure 2 shown, this embodiment also provides a system for implementing a non-driving behavior risk protection method based on driving posture recognition as in Embodiment 1, including: A camera module for acquiring driver video data; A calculation module for executing the non-driving behavior risk protection method based on driving posture recognition as in Embodiment 1 above; An actuator for performing actions according to the safety protection strategy output by the calculation module, including a horn, a seat, and a steering wheel.

[0036] Preferably, the system further includes: a display module for displaying the data processing results of the calculation module.

[0037] After the system starts running, first, the driver video data is acquired through the camera, and then this data enters the calculation module through data transmission. Through the deployed and trained matrix non-driving behavior risk degree evaluation algorithm, the corresponding risk degree level scoring and corresponding countermeasures can be obtained. The corresponding postures and risk degree parameters, etc. will be displayed on the display module, and the actuator will perform corresponding execution actions according to the signals given by the countermeasures.

[0038] The software interface layout of this system can be: the upper left corner is the image display area, the right area is the human posture related parameter display area, the lower left area is the screenshot of the driving posture with a high risk degree level and its parameters, as well as the countermeasures based on the risk degree level, etc., and the bottom area is the operation tool menu bar.

[0039] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art shall fall within the protection scope determined by the claims.

Claims

1. A non-driving behavior risk protection method based on driving posture recognition, characterized in that: The following steps are involved: receiving driver video data; Performing joint point recognition on the driver video data through a vision-based human posture recognition system to obtain the driver's driving posture at each moment; Compare the similarity between the driver's driving posture at each moment and the preset standard driving posture, calculate the degree of deviation of the driving posture at each moment, and score the safety of the degree of deviation of non-driving behavior according to the degree of deviation; The duration of the non-driving behavior is determined by comparing the similarity comparison result with the preset posture similarity threshold, and the corresponding time to leave the position is determined according to the preset safety measures reserved time. The time to leave the position is compared with the human body reaction time to perform a safety score for the time to leave the position for non-driving behavior; A matrix scoring method is used to comprehensively consider the safety scoring results of the degree of departure and the corresponding safety scoring results of the departure time, determine the risk level scoring results of non-driving behavior, and then implement the corresponding safety protection strategy.

2. The non-driving behavior risk protection method based on driving posture recognition according to claim 1 is characterized in that: The degree of displacement is calculated according to the head deflection ratio and joint deviation ratio in the similarity comparison result, and the corresponding calculation expression is: Deviation= + Where Deviation is the overall driving posture deviation, is the calculation weight coefficient of the head deflection ratio to the overall driving posture deviation, is the head deflection ratio, is the calculation weight coefficient of the deviation ratio of each joint to the overall driving posture deviation, The joint deviation ratio.

3. The non-driving behavior risk protection method based on driving posture recognition according to claim 2 is characterized in that: The scoring process of the overall driving posture out-of-position degree is specifically as follows: Pre-divide the range of out-of-position degrees corresponding to different safety levels, and set the corresponding score for each range; According to the interval to which the calculated degree of deviation belongs, a corresponding score value is obtained.

4. The non-driving behavior risk protection method based on driving posture recognition according to claim 1 is characterized in that: The calculation process of the off-position time is specifically as follows: According to the similarity comparison result and the preset posture similarity threshold, if it exceeds the posture similarity threshold, the timing starts to determine the duration of the non-driving behavior, and the corresponding departure time is calculated in combination with the current driving scene and the reserved time for safety measures. The corresponding calculation expression is: T= Where T is the time of leaving the position, is the duration of non-driving behavior, is the scene coefficient, Allow time for safety measures.

5. The non-driving behavior risk protection method based on driving posture recognition according to claim 1 is characterized in that: The scoring process of the off-position time is specifically as follows: Pre-divide the departure time intervals corresponding to different safety levels and set the scores corresponding to each interval; According to the off-location time interval to which the calculated off-location time belongs, a corresponding score value is obtained.

6. The non-driving behavior risk protection method based on driving posture recognition according to claim 1 is characterized in that: The process of determining the non-driving behavior risk level scoring result by the matrix scoring method is specifically as follows: The safety score result of the degree of departure is multiplied by the safety score result of the departure time to obtain the non-driving behavior risk level score result.

7. The non-driving behavior risk protection method based on driving posture recognition according to claim 1 is characterized in that: The execution judgment process of the security protection strategy includes: Pre-dividing the non-driving behavior risk level scoring results into intervals corresponding to different risk levels, wherein the risk levels include high risk, relatively high risk, medium risk and low risk; If it is a high risk, the corresponding response measure is vehicle takeover; If the risk is high, the corresponding response is to request vehicle takeover; If it is a medium risk, the corresponding response measure is to warn of the risk; If the risk is low, the corresponding response measure is no prompt.

8. The non-driving behavior risk protection method based on driving posture recognition according to claim 7 is characterized in that: The risk warning methods include voice warning through a speaker and vibration warning through a seat drive structure.

9. A system for implementing a non-driving behavior risk protection method based on driving posture recognition as described in any one of claims 1 to 8, characterized in that: include: A camera module for acquiring driver video data; A computing module, configured to execute a non-driving behavior risk protection method based on driving posture recognition as described in any one of claims 1 to 8; The actuator is used to execute actions according to the security protection strategy output by the computing module.

10. The system according to claim 9, characterized in that The system further comprises: The display module is used to display the data processing results of the calculation module.

Images