A prediction method for the driver's situational awareness level during L3 driving takeover

Abstract

This application is about a method for predicting the situational awareness level of a driver during an L3 driving takeover, including dividing the takeover road into three sections based on environmental factors and the current task situation; obtaining the environmental factor probability, relative importance of environmental factors, situational awareness prediction probability, and environmental complexity of each section; obtaining the situational awareness level of a single section based on the environmental factor probability, relative importance of environmental factors, situational awareness prediction probability, and environmental complexity, and summing them up to obtain the situational awareness level of the driver on the takeover road. Based on the prediction method, the predicted value of the driver's situational awareness level during the takeover process is obtained, so that the autonomous driving system can better understand the driver's alertness level in different sections of the road and issue takeover requests in a timely manner, thereby reducing traffic accidents caused by driving operation errors; further realizing intelligent cooperation between the autonomous driving system and the driver, and conducting intelligent and humanized interaction.

Description

Technical Field

[0001] The present application relates to the field of traffic safety and driving behavior technology, and in particular to a method for predicting the situational awareness level of a driver taking over at level 3 driving. Background Art

[0002] With the development of autonomous driving technology, autonomous vehicles are gradually becoming the development trend of future transportation. According to the definition of the American SAE, autonomous driving is divided into 6 levels, from L0 (fully manual) to L5 (fully automatic). Among them, L3 level is conditional automation. Under certain conditions, the system completes all driving operations, and the driver provides adaptive responses based on the system's requests. L3 allows the driver to take their hands off the wheel, but this level still requires the driver to control the vehicle. The driver must also stay focused at all times and take over the vehicle when the system is unable to perform the task.

[0003] Therefore, the driver's takeover behavior and situational awareness level at the L3 level are crucial to ensuring driving safety. When the driver takes over the autonomous driving system, the level of his situational awareness will directly affect the safety and response speed of the takeover process.

[0004] When performing driving tasks, drivers will show different levels of situational awareness due to various environmental factors on the road and the various tasks they perform. Therefore, a method is needed to quantify the driver's situational awareness level during the takeover process and evaluate the driver's situational awareness level. Summary of the Invention

[0005] In order to solve or partially solve the problems existing in the related technologies, the present application provides a method for predicting the situational awareness level of the driver during L3 driving takeover, which quantifies the driver's situational awareness level during the takeover process based on the driver's visual cognitive behavioral characteristics.

[0006] In a first aspect, the present application provides a method for predicting the situational awareness level of a driver during an L3 driving takeover, comprising the following steps:

[0007] According to environmental factors and current mission conditions, the takeover road is divided into three sections, including the takeover preparation section, the reaction decision section and the stability control section;

[0008] The environmental factor probability, relative importance of environmental factors, predicted probability of situational awareness, and environmental complexity of each section are obtained respectively. The probability of environmental factors is represented by the probability of fixation, the relative importance of environmental factors is calculated by the importance of environmental factors and the recognition of environmental factors, the predicted probability of situational awareness is the ratio of reaction time to the lead time of takeover request, and the complexity of the environment is represented by the driver's pupil size.

[0009] The situational awareness level of a single section is obtained based on the probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness and the complexity of the environment, and the situational awareness level of the driver who takes over the road is obtained by summing them up.

[0010] Among them, the situational awareness level of the driver who takes over the road is expressed as:

[0011] SA=SA1+SA2+SA3;

[0012] Where SA1 is the situational awareness level of the takeover preparation stage, SA2 is the situational awareness level of the response decision stage, and SA3 is the situational awareness level of the stable control stage.

[0013] The situational awareness level SAi of a single segment is expressed as:

[0014] SA i =β i V i / A i C i ;

[0015] Where βi is the probability of environmental factors, Vi is the relative importance of environmental factors, Ai is the predicted probability of situational awareness, and Ci is the complexity of the environment.

[0016] Among them, the relative importance of environmental factors is expressed as:

[0017]

[0018] Where ei is the importance of environmental factors. The ei of the takeover preparation stage, reaction decision stage, and stable control stage are 0.29, 0.54, and 0.17, respectively. The degree of awareness of environmental factors is calculated using the fuzzy entropy theory.

[0019] in, Expressed as:

[0020]

[0021] Where h i =-e i lne i -(1-e i )ln(1-e i ), hi is the information entropy of ei.

[0022] Among them, A i Expressed as:

[0023]

[0024] Where, t 提前The takeover request advance time is 7 seconds.

[0025] Among them, Ci is described by the ratio of the mean pupil diameter of each segment to the mean sum of the pupil diameters of each segment, which is expressed as:

[0026]

[0027] Where MPZi is the mean pupil diameter at stage i.

[0028] The technical solution provided by this application may have the following beneficial effects:

[0029] This application provides a method for predicting the situational awareness level of a driver during L3 driving takeover, which can systematically analyze the situational awareness level of the driver during the driving takeover process, help realize related research on the analysis of the driver's situational awareness during the driving takeover process, and obtain the predicted value of the driver's situational awareness level during the takeover process based on the driver's eye movement indicators measured during the takeover process. By predicting the driver's situational awareness level, the autonomous driving system can better understand the driver's alertness level in different road sections and issue takeover requests in a timely manner, thereby reducing traffic accidents caused by driving operation errors; further realize the intelligent cooperation between the autonomous driving system and the driver, conduct handovers in a timely manner, avoid unnecessary takeover conflicts, and improve the driver's trust in the autonomous driving system, thereby improving the driving experience; enable the autonomous driving system to conduct intelligent and humanized interaction according to the driver's status, avoid over-reliance on driving operations, and reduce driver fatigue and stress.

[0030] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0032] Figure 1 1 is a flow chart of a prediction method according to an embodiment of the present application;

[0033] Figure 2 2 is a schematic diagram comparing situational awareness levels in the takeover preparation phase of the prediction method shown in an embodiment of the present application;

[0034] Figure 3 2 is a schematic diagram comparing situational awareness levels in the reaction decision-making stage of the prediction method shown in an embodiment of the present application;

[0035] Figure 42 is a schematic diagram comparing situational awareness levels in the stable control phase of the prediction method shown in an embodiment of the present application;

[0036] Figure 5 Schematic diagram of the speed of change of situational awareness at each stage of the prediction method shown in the embodiment of the present application. DETAILED DESCRIPTION

[0037] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although the accompanying drawings illustrate embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0038] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0039] In the description of this application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this application.

[0040] Unless otherwise expressly specified or limited, terms such as "mounted," "connected," "connect," and "fixed" should be interpreted broadly. For example, they may refer to fixed or detachable connections, or integration; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of these terms in this application based on the specific circumstances.

[0041] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0042] Example 1

[0043] like Figure 1 The method for predicting the situational awareness level of a driver taking over an L3 driving position shown includes the following steps:

[0044] S1. Based on environmental factors and current mission conditions, the takeover road is divided into three sections, including the takeover preparation section, the reaction decision section, and the stability control section.

[0045] When performing driving tasks, drivers will display different levels of situational awareness depending on various environmental factors on the road and the various tasks they are performing. Therefore, the entire road can be divided into n sections according to actual conditions, and the situational awareness level of each section can be studied separately.

[0046] Based on the overall division of the takeover process in this paper, it is divided into the takeover preparation stage, the reaction decision stage and the stable control stage.

[0047] Takeover preparation: The driver begins to perceive signals or warnings that the autonomous driving system may need to take over. The driver realizes that the autonomous driving system may not be able to respond effectively to certain road conditions or that a system malfunction has occurred. During this period, the driver focuses on road conditions and system displays, preparing to respond appropriately.

[0048] Reactive Decision-Making Phase: Once the driver realizes the need to take over, the reactive decision-making phase begins. During this phase, the driver must quickly determine whether, when, and how to take over. The driver may need to make decisions based on factors such as road and traffic conditions, and prepare to take action to take control of the vehicle.

[0049] Stable Control Phase: Once the decision to take over driving is made, the stable control phase begins. During this phase, the driver takes actual control of the vehicle, ensuring safety and stability. The driver must adapt to the changes brought about by the switch from the automated driving system to manual driving mode, maintaining control of the vehicle, obeying traffic rules, and maintaining vehicle stability. This phase requires the driver's concentration and precise operation to ensure a safe takeover.

[0050] S2. Obtain the environmental factor probability, relative importance of environmental factors, situational awareness prediction probability and environmental complexity of each section respectively.

[0051] βi is the probability of occurrence of environmental factors, which is represented by the driver's gaze probability.

[0052] Vi is the relative importance of environmental factors in the environment. Here, ei represents the importance of environmental factors in the current driving task. A post-experiment survey of participants revealed their evaluation of the importance of the three stages of the takeover process. Using the entropy weight method, we determined that the ei values ​​for the takeover preparation phase, the reaction decision phase, and the stable control phase were 0.29, 0.54, and 0.17, respectively. Reflects the driver's cognitive state of environmental elements. Since the relative importance is affected by the driver's subjective feelings, it is fuzzy, so the fuzzy entropy theory is used for calculation:

[0053] h i =-e i lne i -(1-e i )ln(1-e i )

[0054]

[0055] In the formula: hi——information entropy of ei;

[0056] ei - the importance of environmental elements in the current driving task;

[0057] ——The driver’s awareness of environmental elements.

[0058] Ai is the driver's prediction efficiency based on situational awareness, that is, the efficiency of the driver's prediction based on situational awareness. Since the application scenario of this study is manual driving, A i It is defined as the ratio of actual speed to design speed. In order to make the model suitable for the study of autonomous driving, this study uses A i Described by the ratio of the reaction time to the takeover request lead time (7 seconds):

[0059]

[0060] Complexity (Ci) is an indicator used to describe the difficulty of understanding environmental information for the driver. In eye movement metrics, pupil size can reflect the driver's cognitive load when processing information. Therefore, Ci is described by the ratio of the mean pupil diameter of each segment to the sum of the mean pupil diameters of all segments, that is:

[0061]

[0062] Where: MPZi - the mean pupil diameter in the i-th stage.

[0063] S3. Based on the probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness, and the complexity of the environment, the situational awareness level of each section is obtained, and the sum is used to obtain the situational awareness level of the driver who takes over the road.

[0064] The calculation process for predicting the situational awareness level of a certain driving task is as follows:

[0065]

[0066] Where: SAi - the situational awareness level of a certain segment;

[0067] n——The number of sections included in the entire road.

[0068] In this embodiment, n=3 in formula (1), and the situational awareness level of the entire takeover process is as follows:

[0069] SA=SA1+SA2+SA3

[0070] Where: SA1 - situational awareness level during takeover preparation;

[0071] SA2 – situational awareness level in the reaction decision segment;

[0072] SA3 - Situational awareness level in the stable control segment.

[0073] For a single segment, the driver's situational awareness level SAi can be obtained by the following formula:

[0074] SA i =β i V i / A i C i

[0075] Where: βi is the probability of the environmental factor appearing in the current task; Vi is the relative importance of the environmental factor in the environment; Ai is the efficiency of the driver's prediction based on situational awareness;

[0076] Ci——the complexity of the environment.

[0077] In summary, the driver's situational awareness level during the takeover process can be predicted based on the driver's eye movement indicators measured during the takeover process.

[0078] Example 2

[0079] The implementation scenario is a two-way, two-lane highway with a speed limit of 120 kilometers per hour. During the autonomous driving process, the vehicle travels at a speed of 100 kilometers per hour and will autonomously change lanes according to the traffic flow ahead, with random traffic density. The experimental scenarios include emergency takeover scenarios and non-emergency takeover scenarios. The main difference between these two situations is that the driver must respond to the TOR in an emergency takeover scenario and take evasive action immediately to avoid a rear-end collision, while non-emergency situations have a higher tolerance for the driver's corresponding TOR, and even if evasive action is not taken immediately, a rear-end collision will not occur. It can be specifically explained as follows:

[0080] 1) In an emergency takeover scenario, due to an accident ahead, the Auto Cruise Control system is unable to interpret the complex traffic conditions and automatically decelerates. It then issues a TOR (Torrent) to the driver, instructing them to retake control immediately. The driver must retake control as quickly as possible and safely avoid the accident ahead.

[0081] 2) As a control, the second experimental scenario involved a non-emergency takeover scenario. This explored the driver's ability to take control when resuming control in normal, non-emergency situations. In this non-emergency takeover scenario, autonomous and non-autonomous driving zones were defined. Once the vehicle exited the autonomous driving zone, the autonomous driving function automatically deactivated, requiring the driver to take control. In this scenario, the driver was required to steer the vehicle safely out of the non-autonomous driving zone.

[0082] 3) The TOR consists of two parts: the vehicle's HMI displays the takeover request and simultaneously issues a voice message. In an emergency, the message reads, "Sudden accident ahead, please take over immediately." In a non-emergency, the message reads, "No autonomous driving zone ahead, please take over." The TOR is issued at the same time.

[0083] Ten seconds after the autonomous driving mode is activated, the driver begins performing non-driving tasks, each of which lasts approximately two minutes before the driver begins the takeover process. Non-driving tasks utilize cognitive, visual, auditory, motor, and a combination of sensory modalities. This study categorizes non-driving tasks into the following three groups:

[0084] Group 1 (Monitoring Group): Autonomous Driving + No-Mission + TOR + Manual Driving

[0085] Group 2 (Music Group): Autonomous Driving + Listening to Music + TOR + Manual Driving

[0086] Group 3 (Video Group): Autonomous Driving + Watching Videos + TOR + Manual Driving

[0087] The monitoring group served as a control group and did not perform the secondary task. The experimental groups listened to music (occupying the auditory channel) and watched videos (occupying both the visual and auditory channels). The effects of different levels of non-driving task involvement on the takeover process were tested. To prevent learning effects, the order of the takeover scenario and the non-driving task was randomized.

[0088] The predicted values ​​of the driver's situational awareness state in all experimental scenarios are calculated. The specific data can be found in Table 1.

[0089] Table 1 Mean predicted values ​​of driver situational awareness level in each situation

[0090]

[0091] Table 1 shows the level of situational awareness at the end of each phase, from highest to lowest: non-emergency monitoring takeover, non-emergency music takeover, non-emergency video takeover, emergency monitoring takeover, emergency music takeover, and emergency video takeover. The results indicate that drivers acquire less situational awareness during the takeover preparation phase due to the shorter timeframe and potential distraction. In contrast, drivers primarily acquire situational awareness during the reaction decision phase. This is because during this phase, drivers face the transition from automatic to manual control, requiring them to quickly acquire environmental information to make correct decisions. During the stable control phase, drivers are engaged in normal manual driving, with the primary goal being to maintain stable driving. Consequently, their situational awareness is more stable.

[0092] The correlation analysis between the theoretical calculation results of the prediction method and the experimental measurement results during the entire takeover process was carried out. The prediction method results were significantly positively correlated with the SART scale score (r=0.565, p=0.008), verifying the effectiveness of the model.

[0093] The level of situational awareness during the takeover preparation phase is as follows: Figure 2As shown in the figure, the data distribution conformed to a normal distribution, so a two-way repeated measures analysis of variance was used for variance analysis. The results showed a significant interaction between takeover scenario and non-driving-related task type on situational awareness levels (F(2,48)=4.857, p=0.011). A simple effects analysis was performed. The results showed that in non-emergency takeover scenarios, situational awareness levels were significantly higher in the monitoring task than in the video task (p=0.000). The situational awareness levels were also significantly higher in the music task than in the video task (p=0.000). When performing the monitoring task, the situational awareness level in the non-emergency takeover scenario was higher than that in the emergency takeover scenario, and the difference was marginally significant (p=0.000); when performing the music task, the situational awareness level in the non-emergency takeover scenario was higher than that in the emergency takeover scenario, and the difference was statistically significant (p=0.000); in the video music task, the situational awareness level in the non-emergency takeover scenario was higher than that in the emergency takeover scenario, and the difference was marginally significant (p=0.077).

[0094] For the reaction decision stage, the situation awareness level is as follows Figure 3 As shown in the figure, the data distribution conformed to a normal distribution, so a two-way repeated measures ANOVA was used for variance analysis. The results showed that the interaction between takeover scenario and non-driving task type on situational awareness levels was not significant (F(2,48)=0.854, p=0.367). A main effects analysis was performed. The differences in situational awareness levels between different non-driving tasks were statistically significant (F(2,48)=3.229, p=0.047), as were the differences in situational awareness levels between different scenarios (F(1,24)=81.237, p=0.000). Bonferroni post hoc tests showed that the situational awareness levels of drivers in emergency takeover scenarios were 1.554 points lower than those in non-emergency takeover scenarios (p=0.006). The situational awareness levels of drivers whose non-driving tasks were video-based were 0.646 points lower than those in the monitoring scenario (p=0.033).

[0095] For the stable control phase, the situational awareness level is as follows Figure 4 As shown in the figure, the data distribution conformed to a normal distribution, so a two-way repeated measures ANOVA was used for variance analysis. The results showed that the interaction between takeover scenario and non-driving task type on situational awareness levels was not significant (F(2,48)=1.279, p=0.285). A main effects analysis was performed. The differences in situational awareness levels between different non-driving tasks were not statistically significant (F(2,48)=0.353, p=0.573), nor were the differences in situational awareness levels between different scenarios statistically significant (F(1,24)=0.827, p=0.370).

[0096] The results showed that, among non-driving tasks, the video task produced the worst SA, suggesting that even though drivers engage in fewer sensory modalities during NDRTs, SA recovery is more difficult. Regarding takeover scenarios, SA recovery levels were higher in non-emergency takeover situations than in emergency situations. This suggests that events requiring urgent attention can weaken drivers' SA levels after takeover. Furthermore, the effects of non-driving tasks and takeover scenarios on situational awareness were limited to the takeover preparation and response decision stages, with no significant impact on stable control.

[0097] Since there is a certain gap in the length of each stage, studying the speed of change of situational awareness can provide a deeper understanding of the changes in situational awareness. Figure 5 The figure shows the recovery speed of situational awareness under different takeover conditions. It shows that the driver's situational awareness recovers fastest during the reaction and decision phase, and during this phase, recovery is significantly slower for the video task in the emergency takeover scenario. Furthermore, it is generally observed that the driver's situational awareness changes slightly slower during the stable control phase than during the takeover preparation phase. However, the speed of change in situational awareness for the non-emergency takeover video task during the stable control phase is greater than during the takeover preparation phase, reaching its maximum during this phase. During the takeover preparation phase, recovery speed for the music task in the non-emergency takeover scenario is fastest, but the difference is not significant. Performing non-driving tasks that occupy the visual senses during the emergency takeover scenario is significantly detrimental to the recovery of situational awareness.

[0098] Finally, it should be noted that, in this document, relationships such as first and second, etc., are used solely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms include, comprise, or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0099] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0100] The embodiments of the present application have been described above. The above description is illustrative and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for predicting the situational awareness level of a driver taking over at level 3 driving, characterized in that: The following steps are involved: According to environmental factors and current mission conditions, the takeover road is divided into three sections, including the takeover preparation section, the reaction decision section and the stability control section; Obtaining, for each of the sections, the probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness, and the degree of environmental complexity, respectively. The probability of environmental factors is represented by the probability of fixation, the relative importance of environmental factors is calculated by the importance of environmental factors and the degree of environmental factor cognition, the predicted probability of situational awareness is the ratio of reaction time to takeover request lead time, and the degree of environmental complexity is represented by the driver's pupil size; The situational awareness level of a single section is obtained based on the probability of environmental factors, the relative importance of environmental factors, the predicted probability of situational awareness and the complexity of the environment, and the situational awareness level of the driver who takes over the road is obtained by summing them up.

2. The method for predicting the situational awareness level of a driver taking over L3 driving according to claim 1 is characterized in that: The driver's situational awareness level of the takeover road is expressed as: SA=SA1+SA2+SA3; Wherein, SA1 is the situational awareness level of the takeover preparation stage, SA2 is the situational awareness level of the reaction decision stage, and SA3 is the situational awareness level of the stable control stage.

3. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 1, characterized in that: The situational awareness level SAi of the single segment is expressed as: SA i =β i In i / A i C i ; Wherein, βi is the probability of the environmental factor, Vi is the relative importance of the environmental factor, Ai is the predicted probability of situational awareness, and Ci is the complexity of the environment.

4. The method for predicting the situational awareness level of a driver taking over at level 3 driving according to claim 1, characterized in that: The relative importance of the environmental factors is expressed as: Where, ei is the importance of the environmental factor. The ei of the takeover preparation stage, reaction decision stage, and stable control stage are 0.29, 0.54, and 0.17, respectively. The degree of awareness of the environmental factors is calculated using fuzzy entropy theory.

5. The method for predicting the situational awareness level of a driver taking over at level L3 according to claim 4, characterized in that: described Expressed as: Where h i =-e i lne i -(1-e i )ln(1-e i ), hi is the information entropy of ei.

6. The method for predicting the situational awareness level of a driver taking over at level 3 driving according to claim 3 is characterized in that: The A i Expressed as: In the formula, the t 提前 The takeover request advance time is 7 seconds.

7. The method for predicting the situational awareness level of a driver taking over at level 3 driving according to claim 3 is characterized in that: The Ci is described by the ratio of the mean pupil diameter of each segment to the mean sum of the pupil diameters of each segment, expressed as: Where MPZi is the mean pupil diameter at stage i.

Images