Method for predicting scene awareness level of L3-level driving takeover driver

Abstract

The invention relates to a method for predicting the consciousness level of an L3-level driving takeover driver, and the method comprises the steps: dividing a takeover road into three sections according to environment factors and a current task condition; respectively acquiring the environmental factor probability, the environmental factor relative importance, the scene awareness prediction probability and the environmental complexity of each section; and based on the environmental factor probability, the environmental factor relative importance, the scene awareness prediction probability and the environmental complexity, obtaining the scene awareness level of a single section, and summing to obtain the scene awareness level of the driver of the takeover road. The situation awareness level prediction value of the driver in the takeover process is obtained based on the prediction method, so that the automatic driving system can better know the alertness degree of the driver in different road sections and send out a takeover request in time, and therefore traffic accidents caused by driving misoperation are reduced; and intelligent cooperation between the automatic driving system and the driver is further realized, and intelligent and humanized interaction is carried out.

Description

Technical Field

[0001] The present application relates to the field of traffic safety and driving behavior technology, and in particular to a method for predicting the situational awareness level of a driver taking over at level L3 driving. Background Art

[0002] With the development of autonomous driving technology, autonomous vehicles are gradually becoming the development trend of future transportation. According to the definition of SAE in the United States, autonomous driving is divided into 6 levels, from L0 (fully manual) to L5 (fully automatic). Among them, L3 level is conditional automation. Under certain conditions, the system completes all driving operations, and the driver provides adaptive responses according to the system request. L3 allows the driver to take his hands off the wheel, but this level still requires the driver to control it. The driver must also stay focused at all times and take over the vehicle when the system cannot perform the task.

[0003] Therefore, the driver's takeover behavior and situational awareness level at the L3 level are crucial to ensuring driving safety. When the driver takes over the autonomous driving system, the level of his or her situational awareness will directly affect the safety and response speed of the takeover process.

[0004] When performing driving tasks, drivers will show different levels of situational awareness due to various environmental factors on the road and the various tasks they perform. Therefore, a method is needed to quantify the driver's situational awareness level during the takeover process and to evaluate the driver's situational awareness level. Summary of the invention

[0005] In order to solve or partially solve the problems existing in the related technology, the present application provides a method for predicting the situational awareness level of the driver during L3 driving takeover, which quantifies the driver's situational awareness level during the takeover process based on the driver's visual cognitive behavioral characteristics.

[0006] The first aspect of the present application provides a method for predicting the situational awareness level of a driver taking over at level L3 driving, comprising the following steps:

[0007] According to environmental factors and current mission conditions, the takeover road is divided into three sections, including the takeover preparation section, the reaction decision section and the stability control section;

[0008] The probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness and the complexity of the environment are obtained for each section respectively. The probability of environmental factors is represented by the probability of fixation, the relative importance of environmental factors is calculated by the importance of environmental factors and the recognition of environmental factors, the predicted probability of situational awareness is the ratio of reaction time to the lead time of takeover request, and the complexity of the environment is represented by the size of the driver's pupil.

[0009] The situational awareness level of a single section is obtained based on the probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness and the complexity of the environment, and the situational awareness level of the driver who takes over the road is obtained by summing them up.

[0010] Among them, the situational awareness level of the driver who takes over the road is expressed as:

[0011] SA=SA1+SA2+SA3;

[0012] Where SA1 is the situational awareness level of the takeover preparation stage, SA2 is the situational awareness level of the reaction decision stage, and SA3 is the situational awareness level of the stable control stage.

[0013] The situational awareness level SAi of a single segment is expressed as:

[0014] S i =β i V i / A i C i ;

[0015] Where βi is the probability of environmental factors, Vi is the relative importance of environmental factors, Ai is the predicted probability of situational awareness, and Ci is the complexity of the environment.

[0016] Among them, the relative importance of environmental factors is expressed as:

[0017]

[0018] In the formula, ei is the importance of environmental factors, and the ei of the takeover preparation stage, reaction decision stage, and stable control stage are 0.29, 0.54, and 0.17, respectively; The degree of environmental factor cognition is calculated based on fuzzy entropy theory.

[0019] in, It is expressed as:

[0020]

[0021] In the formula, h i =-e i lne i -(1-e i )ln(1-e i ), hi is the information entropy of ei.

[0022] Among them, A i It is expressed as:

[0023]

[0024] In the formula, t 提前The lead time for taking over the request is 7 seconds.

[0025] Among them, Ci is described by the ratio of the mean pupil diameter of each segment to the mean sum of the pupil diameters of each segment, expressed as:

[0026]

[0027] Where MPZi is the mean pupil diameter in the i-th stage.

[0028] The technical solution provided by this application may have the following beneficial effects:

[0029] The present application provides a method for predicting the situational awareness level of a driver during L3 driving takeover, which can systematically analyze the situational awareness level of the driver during the driving takeover process, help realize the relevant research on the situational awareness analysis of the driver during the driving takeover process, and obtain the predicted value of the situational awareness level of the driver during the takeover process based on the driver's eye movement indicators measured during the takeover process. By predicting the driver's situational awareness level, the autonomous driving system can better understand the driver's alertness level in different sections of the road, and issue takeover requests in a timely manner, thereby reducing traffic accidents caused by driving operation errors; further realize the intelligent cooperation between the autonomous driving system and the driver, conduct handovers in a timely manner, avoid unnecessary takeover conflicts, and increase the driver's trust in the autonomous driving system, thereby improving the driving experience; enable the autonomous driving system to interact intelligently and humanely according to the driver's status, avoid over-reliance on driving operations, and reduce driver fatigue and stress.

[0030] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0032] Figure 1 It is a schematic diagram of the flow of the prediction method shown in the embodiment of the present application;

[0033] Figure 2 It is a schematic diagram comparing the situational awareness levels in the takeover preparation stage of the prediction method shown in the embodiment of the present application;

[0034] Figure 3 It is a schematic diagram comparing the situational awareness levels in the reaction decision stage of the prediction method shown in the embodiment of the present application;

[0035] Figure 4is a schematic diagram comparing situational awareness levels in the stable control phase of the prediction method shown in an embodiment of the present application;

[0036] Figure 5 It is a schematic diagram of the speed of change of situational awareness at each stage of the prediction method shown in the embodiment of the present application. DETAILED DESCRIPTION

[0037] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0038] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0039] In the description of the present application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0040] Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0041] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0042] Embodiment 1

[0043] like Figure 1 The method for predicting the situational awareness level of a driver taking over a Level 3 driving operation shown in the figure comprises the following steps:

[0044] S1. According to environmental factors and current mission conditions, the takeover road is divided into three sections, including the takeover preparation section, the reaction decision section and the stable control section.

[0045] When performing driving tasks, drivers will show different levels of situational awareness due to various environmental factors on the road and the various tasks they perform. Therefore, the entire road can be divided into n sections according to actual conditions, and the situational awareness level of each section can be studied separately.

[0046] Based on the overall division of the takeover process in this article, it is divided into the takeover preparation stage, the reaction decision stage and the stable control stage.

[0047] Takeover preparation phase: The driver begins to perceive signals or warnings that the autonomous driving system may need to take over. The driver will realize that the autonomous driving system may not be able to cope well with certain road conditions, or that the system has failed. In the takeover preparation phase, the driver will begin to focus on the road conditions and the system display information, and prepare to respond appropriately.

[0048] Reaction decision stage: Once the driver realizes that he needs to take over driving, the reaction decision to take over begins. At this stage, the driver needs to make a quick judgment to decide whether to take over driving, when to take over, and how to take over. The driver may need to make decisions based on factors such as road conditions and traffic conditions, and be prepared to take action to take over control of the vehicle.

[0049] Stable control stage: Once the decision to take over driving is made, the stable control stage of takeover begins. In this stage, the driver begins to actually control the vehicle to ensure the safety and stability of the vehicle. The driver needs to adapt to the changes brought about by the switch from the automatic driving system to the manual driving mode, and pay attention to synchronously controlling the vehicle, complying with traffic rules and maintaining vehicle stability. In this stage, the driver needs to concentrate and make accurate operations to ensure a safe takeover process.

[0050] S2. Obtain the probability of environmental factors, relative importance of environmental factors, predicted probability of situational awareness and environmental complexity of each section respectively.

[0051] βi is the probability of occurrence of environmental factors, and the probability of environmental factors is represented by the driver's gaze probability.

[0052] Vi is the relative importance of environmental factors in the environment. Among them, ei represents the importance of environmental elements in the current driving task. The subjects were surveyed after the experiment, and according to their evaluation of the importance of the three stages in the takeover process, the entropy weight method was used to obtain the ei of the takeover preparation stage, reaction decision stage, and stable control stage to be 0.29, 0.54, and 0.17, respectively. It reflects the driver's cognitive state of environmental elements. Since the relative importance is affected by the driver's subjective feelings, it is fuzzy, so the fuzzy entropy theory is used for calculation:

[0053] h i =-e i lne i -(1-e i )ln(1-e i )

[0054]

[0055] In the formula: hi——information entropy of ei;

[0056] ei - the importance of environmental elements in the current driving task;

[0057] ——The driver’s awareness of environmental elements.

[0058] Ai is the driver's prediction efficiency based on situational awareness, that is, the efficiency of the driver's prediction based on situational awareness. Since the scenario applied in this study is manual driving, A i It is defined as the ratio of the actual speed to the design speed. In order to adapt the model to the research of autonomous driving, this study uses A i Described by the ratio of the response time to the takeover request lead time (7 seconds):

[0059]

[0060] The complexity level Ci is an indicator used to describe the difficulty of understanding environmental information for drivers. In the eye movement metric, pupil size can reflect the cognitive load of drivers when processing information. Therefore, Ci is described by the ratio of the mean pupil diameter of each segment to the mean sum of the pupil diameters of each segment, that is:

[0061]

[0062] Where: MPZi is the mean pupil diameter in the i-th stage.

[0063] S3. Based on the probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness and the complexity of the environment, the situational awareness level of a single section is obtained, and the sum is used to obtain the situational awareness level of the driver who takes over the road.

[0064] The calculation process of predicting the situational awareness level of a certain driving task is as follows:

[0065]

[0066] Where: SAi – the situational awareness level of a certain section;

[0067] n – the number of sections included in the entire road.

[0068] In this embodiment, in formula (1), n=3, and the situational awareness level of the entire takeover process is as follows:

[0069] SA=SA1+SA2+SA3

[0070] Where: SA1 – situational awareness level in the takeover preparation phase;

[0071] SA2 – level of situational awareness in the reaction decision segment;

[0072] SA3 – Situational awareness level in the stable control segment.

[0073] For a single segment, the driver's situational awareness level SAi can be obtained by the following formula:

[0074] S i =β i V i / A i C i

[0075] Where: βi - the probability of the environmental factor appearing in the current task; Vi - the relative importance of the environmental factor in the environment; Ai - the efficiency of the driver to make predictions based on situational awareness;

[0076] Ci——the complexity of the environment.

[0077] In summary, the predicted value of the driver's situational awareness level during the takeover process can be obtained based on the driver's eye movement indicators measured during the takeover process.

[0078] Embodiment 2

[0079] The implementation scenario is a two-way two-lane highway with a speed limit of 120 kilometers per hour. During the autonomous driving process, the vehicle travels at a speed of 100 kilometers per hour and will change lanes autonomously according to the traffic flow ahead, with random traffic density. The experimental scenarios include emergency takeover scenarios and non-emergency takeover scenarios. The main difference between these two situations is that the driver must respond to TOR in a timely manner in an emergency takeover scenario and take evasive action immediately to avoid a rear-end collision, while non-emergency situations have a higher tolerance for the driver's corresponding TOR, and even if evasive action is not taken immediately, a rear-end collision will not occur. It can be explained specifically as:

[0080] 1) In the case of an emergency takeover scenario, due to an accident vehicle on the road ahead, the auto cruise system cannot analyze such complex traffic conditions, automatically starts to slow down, and sends a TOR to the driver, instructing the driver to urgently retake control of the vehicle. The driver must retake control of the vehicle as soon as possible and safely avoid the accident vehicle ahead.

[0081] 2) As a control, the second type of experimental scenario is a non-emergency takeover scenario. The takeover situation when the driver resumes control in normal and non-emergency situations is explored. In the non-emergency takeover scenario, the autonomous driving and non-autonomous driving areas are set, and it is stipulated that once the vehicle leaves the autonomous driving area, the autonomous driving function will automatically fail, and the driver is required to take over the control of the vehicle. In this case, the driver needs to control the vehicle and safely leave the non-autonomous driving area.

[0082] 3) TOR consists of two parts. The vehicle HMI displays the takeover request and issues a voice message at the same time. The voice message sent in an emergency is "There is an accident ahead, please take over immediately"; the voice message sent in a non-emergency situation is "There is a non-autonomous driving area ahead, please take over the vehicle." The TOR is issued at the same time.

[0083] After 10 seconds of the automatic driving mode being turned on, the driver begins to perform non-driving tasks, and each time the driver has to go through the automatic driving stage for about 2 minutes before starting the takeover process. The types of sensory modes occupied by non-driving tasks include cognition, vision, hearing, action, and combination. This study divides non-driving tasks into the following three groups:

[0084] Group 1 (monitoring group): autonomous driving + no secondary mission + TOR + manual driving

[0085] Group 2 (Music Group): Autonomous Driving + Listening to Music + TOR + Manual Driving

[0086] Group 3 (Video Group): Autonomous Driving + Watching Video + TOR + Manual Driving

[0087] The monitoring group was selected as the control group, that is, no secondary task was performed. Listening to music (occupying the auditory channel) and watching videos (occupying the visual and auditory channels) were used as the experimental group. The effects of non-driving tasks with different levels of involvement on the takeover process were tested. In order to avoid learning effects, the order of combining the takeover scenario with the non-driving task was randomized.

[0088] The predicted values ​​of the driver's situational awareness state in all experimental scenarios are calculated. The specific data can be viewed in Table 1.

[0089] Table 1 Mean predicted values ​​of driver situational awareness level in each situation

[0090]

[0091] The calculation results in Table 1 are the situational awareness levels at the end of each stage, from the highest to the lowest: monitoring non-emergency takeover, music non-emergency takeover, video non-emergency takeover, monitoring emergency takeover, music emergency takeover, video emergency takeover. The results show that due to the short time and possible distraction effects, the driver's situational awareness is less in the takeover preparation stage. In contrast, the driver mainly acquires situational awareness in the reaction decision stage. This is because in this stage the driver faces a transition from automatic control to manual control, and the driver needs to quickly obtain environmental information to make correct decisions. In the stable control stage, the driver is equivalent to the normal manual driving stage, and the main goal is to maintain stable driving, so the acquisition of situational awareness is also relatively stable.

[0092] The theoretical calculation results of the prediction method in the whole takeover process were correlated with the experimental measurement results. The prediction method results were significantly positively correlated with the SART scale score (r=0.565, p=0.008), which verified the effectiveness of the model.

[0093] The level of situational awareness during the takeover preparation phase is as follows: Figure 2As shown. The data distribution conforms to the normal distribution, so a two-way repeated measures ANOVA was used for difference analysis. The results showed that the interaction between the takeover scenario and the type of non-driving-related task on the level of situational awareness was significant (F(2,48)=4.857, p=0.011), and a simple effect analysis was performed. The results showed that in non-emergency takeover scenarios, the situational awareness level of the monitoring task was higher than that of the video task, and the difference was statistically significant (p=0.000); the situational awareness level of the music task was higher than that of the video task, and the difference was statistically significant (p=0.000). When performing monitoring tasks, the situational awareness level in non-emergency takeover scenarios was higher than that in emergency takeover scenarios, and the difference was marginally significant (p=0.000); when performing music tasks, the situational awareness level in non-emergency takeover scenarios was higher than that in emergency takeover scenarios, and the difference was statistically significant (p=0.000); in video music tasks, the situational awareness level in non-emergency takeover scenarios was higher than that in emergency takeover scenarios, and the difference was marginally significant (p=0.077).

[0094] For the reaction decision stage, the level of situational awareness is Figure 3 As shown. The data distribution conforms to the normal distribution, so a two-way repeated measures ANOVA was used for difference analysis. The results showed that the interaction between the takeover scenario and the type of non-driving-related task on the situational awareness level was not significant (F(2,48)=0.854, p=0.367), and a main effect analysis was performed. The difference in the situational awareness level of different non-driving tasks was statistically significant (F(2,48)=3.229, p=0.047), and the difference in the situational awareness level of different scenarios was statistically significant (F(1,24)=81.237, p=0.000). The results of the Bonferroni post hoc test showed that the situational awareness level of the driver in the emergency takeover scenario was 1.554 lower than that in the non-emergency takeover scenario (p=0.006), and the situational awareness level of the non-driving task video was 0.646 lower than that of the monitoring (p=0.033).

[0095] For the stable control phase, the situational awareness level is Figure 4 As shown. The data distribution conforms to the normal distribution, so a two-way repeated measures ANOVA was used for difference analysis. The results showed that the interaction between the takeover scenario and the type of non-driving related task on the level of situational awareness was not significant (F(2,48)=1.279, p=0.285), and a main effect analysis was performed. The differences in the level of situational awareness of different non-driving tasks were not statistically significant (F(2,48)=0.353, p=0.573), and the differences in the level of situational awareness of different scenarios were not statistically significant (F(1,24)=0.827, p=0.370).

[0096] The results showed that, on the one hand, for non-driving tasks, the SA level of the video task was the worst, indicating that the less sensory modalities the driver engaged in NDRTs, the more difficult it was to recover SA. For takeover scenarios, the SA recovery level in non-emergency takeover situations was higher than that in emergency situations. This suggests that events that require emergency handling will weaken the driver's SA level after taking over. On the other hand, the effects of non-driving tasks and takeover scenarios on situational awareness only exist in the takeover preparation and reaction decision stages, and have no significant effect on the stable control stage.

[0097] Since there is a certain gap in the length of each stage, studying the speed of change of situational awareness can provide a deeper understanding of the change of situational awareness. Figure 5 The recovery speed of situational awareness under different takeover conditions shown in the figure shows that the driver's situational awareness recovers fastest in the reaction decision stage, and in this stage, the video task recovery is significantly slowest in the emergency takeover situation. In addition, it is generally believed that the driver's situational awareness changes slightly slower in the stable control stage than in the takeover preparation stage. However, the situational awareness change speed of the non-emergency takeover video task in the stable control stage is greater than that in the takeover preparation stage, and is the largest in this stage. In the takeover preparation stage, the music task in the non-emergency takeover situation recovers the fastest, but the difference is not obvious. Performing non-driving tasks that occupy the visual senses in an emergency takeover scenario is very detrimental to the recovery of situational awareness.

[0098] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms include, include or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0099] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0100] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for predicting the situational awareness level of a driver taking over at level L3 driving, characterized in that: The following steps are involved: According to environmental factors and current mission conditions, the takeover road is divided into three sections, including the takeover preparation section, the reaction decision section and the stability control section; respectively obtaining the environmental factor probability, relative importance of environmental factors, predicted probability of situational awareness and environmental complexity of each section, wherein the environmental factor probability is represented by the fixation probability, the relative importance of environmental factors is calculated by the importance of environmental factors and the recognition of environmental factors, the predicted probability of situational awareness is the ratio of reaction time to lead time of takeover request, and the environmental complexity is represented by the pupil size of the driver; The situational awareness level of a single section is obtained based on the environmental factor probability, the relative importance of the environmental factor, the situational awareness prediction probability and the environmental complexity, and the situational awareness level of the driver who takes over the road is obtained by summing them up.

2. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 1, characterized in that: The driver's situational awareness level of the takeover road is expressed as: SA=SA1+SA2+SA3; Wherein, SA1 is the situational awareness level of the takeover preparation stage, SA2 is the situational awareness level of the reaction decision stage, and SA3 is the situational awareness level of the stable control stage.

3. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 1, characterized in that: The situational awareness level SAi of the single segment is expressed as: SA i =β i In i / A i C i ; Wherein, βi is the probability of the environmental factor, Vi is the relative importance of the environmental factor, Ai is the predicted probability of situational awareness, and Ci is the complexity of the environment.

4. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 1, characterized in that: The relative importance of the environmental factors is expressed as: In the formula, ei is the importance of the environmental factor, and the ei of the takeover preparation stage, reaction decision stage, and stable control stage are 0.29, 0.54, and 0.17, respectively; The degree of recognition of the environmental factors is calculated by citing the fuzzy entropy theory.

5. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 4, characterized in that: Said It is expressed as: In the formula, h i =-e i lne i -(1-e i )ln(1-e i ), hi is the information entropy of ei.

6. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 3, characterized in that: The A i It is expressed as: In the formula, the t 提前 The advance time for the takeover request is 7 seconds.

7. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 3, characterized in that: The Ci is described by the ratio of the mean pupil diameter of each segment to the mean sum of the pupil diameters of each segment, expressed as: Where MPZi is the mean pupil diameter in the i-th stage.

Images