Method and device for supporting manual driving of a driver of a vehicle

Abstract

The present invention relates to the field of human-computer interaction. The present invention provides a method for supporting manual driving of a driver of a vehicle, wherein the vehicle travels at least partially autonomously, the method comprising the following steps: S1: obtaining degradation information during vehicle driving, wherein the degradation information reflects that the driver manually takes over the driving task from the autonomous mode of the vehicle; S2: providing at least one reward interaction function based on the degradation information, wherein the reward interaction function is configured to motivate the driver to actively exit the autonomous mode of the vehicle and switch to the manual mode through interaction with the driver. The present invention also provides a device for supporting manual driving of a driver of a vehicle, a vehicle, and a machine-readable storage medium. Through the user interaction strategy provided by the present invention, the user will not forget how to drive the vehicle safely and independently even in the presence of a driver assistance system.

Description

Technical Field

[0001] The present invention relates to a method for supporting a driver of a vehicle in manual driving. The present invention also relates to a device for supporting a driver of a vehicle in manual driving, a vehicle, and a machine-readable storage medium. Background Art

[0002] Currently, an increasing number of vehicles are equipped with autonomous driving and driver assistance systems. While these systems can alleviate the burden on drivers to a certain extent, their widespread use is also leading to a degradation of drivers' manual driving skills. If a vehicle's autonomous driving system malfunctions, or if a driver temporarily needs to operate a vehicle without autonomous driving capabilities, their inexperienced manual driving skills can pose a safety hazard to road traffic. Therefore, while vigorously promoting the intelligentization of vehicles, the degradation of users' manual driving skills also deserves attention.

[0003] To this end, the prior art proposes to monitor the driver's behavior when the learning mode of the autonomous driving vehicle is activated, so as to present guidance information to the driver when the driver is not competent for the driving task, or to switch the vehicle back to the autonomous driving mode.

[0004] However, the above solutions still have many shortcomings. In particular, they currently only propose to provide safety assistance to drivers in manual driving mode, but still lack a safe and effective interaction mechanism to encourage drivers to actively abandon the comfortable automatic driving mode and spontaneously exercise their manual driving skills.

[0005] In this context, it is desirable to provide a user interaction strategy for autonomous vehicles so that users do not forget how to drive the vehicle safely and independently even in the presence of a driver assistance system. Summary of the Invention

[0006] The present invention relates to a method for supporting a driver in manual driving of a vehicle, the vehicle being driven at least partially autonomously, comprising the following steps:

[0007] S1: obtaining degradation information during vehicle driving, the degradation information reflecting that a driver manually takes over a driving task from an autonomous mode of the vehicle; and

[0008] S2: Providing at least one reward interaction function based on the degradation information, wherein the reward interaction function is configured to encourage the driver to actively exit the autonomous mode of the vehicle and switch to the manual mode through interaction with the driver.

[0009] The present invention specifically incorporates the following technical concepts: By linking the user's manual driving behavior with a reward mechanism, the reduced comfort caused by abandoning the autonomous driving function is compensated through appropriate interactive methods. This allows the user to no longer experience boredom and fatigue while manually operating the vehicle, but instead experience the fun and challenge brought by the rewarding interactive function, greatly increasing the user's motivation to manually drive the vehicle. Therefore, even if the vehicle is equipped with a driver assistance system or an autonomous driving system, users can be encouraged to spontaneously practice their manual driving skills.

[0010] Optionally, the downgrade information reflects that the driver manually took over the driving task from the vehicle's autonomous mode when there was no reason to take over. By limiting the scenarios in which rewarded interactive functions are provided, users can be further encouraged to spontaneously practice manual driving skills.

[0011] Optionally, step S2 includes: allocating reward points to the driver based on the demotion information; and outputting the number of reward points to the driver visually and / or audibly. This helps the user dynamically understand the changes in their manual driving experience and the accumulation of reward content.

[0012] Optionally, the reward interaction function includes multiple skill levels, and different skill levels are bound to different score thresholds, wherein:

[0013] - when the bonus points allocated to the driver exceed a determined points threshold, outputting a prompt to the driver that the skill level bound to the points threshold is unlocked; and / or

[0014] If the bonus points allocated to the driver do not exceed a predetermined point threshold, a difference between the bonus points and the predetermined point threshold is output to the driver.

[0015] By linking manual driving operations with the improvement of skill levels, users can be attracted to continuously unlock the next level title, thereby increasing their enthusiasm for participating in manual driving.

[0016] Optionally, in step S2, reward points are allocated to the driver based on the difficulty level of the traffic scenario during manual driving, the driver's behavior during manual driving, and the duration of manual driving. This allows for more objective reward results based on various factors, thereby preventing users from feeling overly difficult or easy to advance through the level, which could lead to a decrease in their motivation to participate.

[0017] Optionally, in step S2, during the driver's manual driving of the vehicle, incremental reward points are calculated for each time increment, weighted by the difficulty level of the traffic scenario and the driver's behavioral performance. The incremental reward points for each time increment are accumulated to form the reward points. This allows for a more balanced consideration of the impact of different factors on manual driving motivation and demonstrates the cumulative effect of reward points over time, which helps users see their manual driving skills improve over time and enhances their interest in manual driving.

[0018] Optionally, the degradation information includes the driver's behavior during manual driving of the vehicle, wherein the driver's behavior during manual driving of the vehicle is obtained based on at least one of the following:

[0019] - the degree of agreement between the vehicle's actual driving trajectory and the predefined reference driving trajectory during manual driving by the driver;

[0020] - the number and / or extent to which the vehicle's speed deviates from a predefined reference speed interval during manual driving of the vehicle by the driver;

[0021] - the number of times the driver accelerates and / or brakes beyond a preset magnitude threshold while the driver is manually driving the vehicle;

[0022] -The vehicle's relationship to other surrounding traffic objects during manual driving by the driver.

[0023] By understanding the above information, we can more accurately evaluate the user's operating accuracy and correctness of driving habits during manual driving.

[0024] Optionally, the degradation information includes the difficulty level of the traffic scenario during manual driving by the driver, wherein the difficulty level of the traffic scenario is determined by using a trained machine learning model to determine the difficulty level of the traffic scenario based on weather conditions, road type, traffic volume, and / or the presence of vulnerable road users. The introduction of the machine learning model not only enables a faster and more reliable assessment of the difficulty level of the traffic scenario, but also, through the self-learning capabilities of the machine learning model, a logical relationship between the difficulty level of the traffic scenario and time can be summarized, thereby enabling an estimation of the changing trend of the difficulty of the traffic scenario over time.

[0025] Optionally, the vehicle's autonomous mode includes at least a first autonomous mode and a second autonomous mode, wherein the degradation information further reflects that, especially when there is no reason for degradation, the driver switches from the first autonomous mode of the vehicle to the second autonomous mode, and the automatic driving level corresponding to the second autonomous mode is lower than the automatic driving level corresponding to the first autonomous mode.

[0026] In step S2 , the reward interaction function is further configured to encourage the driver to actively exit the first autonomous mode and switch to the second autonomous mode through interaction with the driver.

[0027] Optionally, in step S2 , the reward level determined for the first autonomous mode of the vehicle is lower than the reward level determined for the second autonomous mode of the vehicle, and the reward level determined for the second autonomous mode of the vehicle is lower than the reward level determined for the manual mode of the vehicle.

[0028] By linking the degradation of autonomous mode to a reward mechanism, drivers can be gradually guided to give up the more comfortable level of automated driving. This allows even drivers with limited manual driving skills or unfamiliarity with manual operation to gradually adapt to manual driving and experience the joy of reward-based interaction. This makes the entire reward mechanism more comprehensive and user-friendly.

[0029] Optionally, the reward interaction function includes a virtual redemption service, configured to present the driver with the following redemption options based on downgrade information: unlocking skill levels, virtual gifts, physical gifts, and / or discounts on vehicle services. This reward redemption system further motivates users to actively participate in manual driving, provides practical use for reward points, and promotes their efficient use and accumulation.

[0030] Optionally, the reward interaction function includes a task collection consisting of multiple game tasks. In step S2, the completion progress of one or more game tasks in the task collection is calculated based on the demotion information, and the completion progress is output to the driver. This enriches the user's manual driving experience, brings more fun, and can use the game task achievement mechanism to encourage users to actively complete more skill challenges.

[0031] Optionally, changes in the driver's manual driving experience and / or manual driving proficiency are identified based on the degradation information; and / or, in response to identifying changes in manual driving experience and / or manual driving proficiency, the triggering method of the reward interaction function and / or the interaction method with the driver are dynamically updated. This helps to flexibly adapt the reward interaction method to the driver's manual driving experience and proficiency, thereby enhancing user engagement at different stages based on their interests and driving proficiency, and avoiding user loss due to unchanging triggering mechanisms.

[0032] Optionally, in step S1, it is determined that the driver manually takes over the driving task from the autonomous mode of the vehicle when the following predefined behaviors of the driver are detected:

[0033] Detecting that at least one of the driver's hands is holding the steering wheel;

[0034] receiving a driver's voice command, gesture command, or touch command on a predefined button to activate manual mode; and / or

[0035] It is detected that at least one foot of the driver is on an accelerator pedal and / or a brake pedal of the vehicle.

[0036] As a result, the user's spontaneous switch to manual driving behavior can be identified more accurately, thereby effectively filtering the scenarios that provide rewarding interactions.

[0037] Optionally, step S1 further includes: in response to detecting a predefined driver behavior, outputting a prompt message to the driver of the vehicle to switch the vehicle to manual mode, wherein the driver's manual takeover of the driving task from the vehicle's autonomous mode is confirmed only if the driver responds affirmatively to the prompt message. This helps the driver confirm whether they truly want to exit the autonomous mode, avoiding erroneous mode switching or reward statistics due to user error.

[0038] According to a second aspect of the present invention, there is provided a device for supporting manual driving of a driver of a vehicle, the device being configured to execute the method according to the first aspect of the present invention, the device comprising:

[0039] an acquisition module configured to acquire degradation information during driving of the vehicle, the degradation information reflecting that a driver manually takes over a driving task from an autonomous mode of the vehicle; and

[0040] A module is provided, which is configured to provide at least one reward interaction function based on degradation information, wherein the reward interaction function is configured to encourage the driver to actively exit the autonomous mode of the vehicle and switch to the manual mode through interaction with the driver.

[0041] According to a third aspect of the present invention, there is provided a vehicle that travels at least partially autonomously, the vehicle comprising an apparatus according to the first aspect of the present invention.

[0042] According to a fourth aspect of the present invention, a machine-readable storage medium is provided, on which a computer program is stored, for executing the method according to the first aspect of the present invention when the computer program is run on a computer. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The present invention will be described in more detail below with reference to the accompanying drawings, so that the principles, features and advantages of the present invention can be better understood. The accompanying drawings include:

[0044] Figure 1 A block diagram illustrating an apparatus for supporting manual driving of a driver of a vehicle according to an exemplary embodiment of the present invention is shown;

[0045] Figure 2 A flow chart showing a method for supporting manual driving of a driver of a vehicle according to an exemplary embodiment of the present invention;

[0046] Figure 3 In one exemplary embodiment, Figure 2 A flow chart of one method step of the method shown;

[0047] Figure 4 In another exemplary embodiment, Figure 2 A flow chart of one method step of the method shown;

[0048] Figure 5 In one exemplary embodiment, Figure 2 A flow chart of another method step of the method shown; and

[0049] Figure 6a 、 Figure 6b 、 Figure 6c 、 Figure 6d A schematic diagram illustrating an interactive interface of a reward interaction function according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0050] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the scope of protection of the present invention.

[0051] Figure 1 A block diagram illustrating an apparatus for supporting manual driving of a driver of a vehicle according to an exemplary embodiment of the present invention is shown.

[0052] like Figure 1 As shown, the vehicle 100 includes a device 1 for supporting the driver's manual driving. The vehicle 100 also includes, for example, a panoramic visual perception system composed of a front-view camera 11, a left-view camera 12, a rear-view camera 13, and a right-view camera 14, a radar sensor, and a lidar sensor. With the help of these vehicle external environment sensors, the vehicle 100 can, for example, perform various functions such as reversing assistance, obstacle detection, and road structure recognition to support partially autonomous driving or fully autonomous driving. In addition, the vehicle 100 also includes a certain number of vehicle internal environment sensors, such as a steering wheel sensor 15, an accelerator pedal sensor 16, a brake pedal sensor 17, and an in-vehicle camera 18. With the help of these sensors, the actions, expressions, and sight directions of the occupants in the vehicle 100 can be detected, thereby enabling the inference of the person's behavioral intentions.

[0053] It should be noted that except for Figure 1 In addition to the vehicle-mounted sensors shown, the vehicle 100 may also include other types and quantities of sensors, which are not specifically limited in the present invention.

[0054] In order to train the manual driving skills of the driver of autonomous vehicle 100 , device 1 comprises, for example, an acquisition module 10 and a provision module 20 , which are connected to one another via communication technology.

[0055] Acquisition module 10 is used to acquire degradation information during vehicle 100's operation, indicating that the driver has manually taken over driving from the vehicle's autonomous mode. To this end, acquisition module 10 is connected to, for example, vehicle 100's steering wheel sensor 15, accelerator pedal sensor 16, brake pedal sensor 17, and in-vehicle camera 18 to detect the driver's behavior and thereby infer whether the driver has indicated an intention to actively exit or downgrade the autonomous driving level of the vehicle's autonomous mode. For example, while vehicle 100 is in autonomous mode, acquisition module 10 can determine whether the driver has intervened in the vehicle's autonomous driving based on the brake pedal position detected by brake pedal sensor 17.

[0056] Once the detection results from the vehicle's internal environmental sensors indicate that the driver has manually taken over the driving task from autonomous mode, the acquisition module 10 can further collect data from the period during which the vehicle 100 was manually guided using other onboard sensors. To this end, the acquisition module 10 can, for example, be connected to the vehicle's external environmental sensors (cameras, lidar sensors, positioning and navigation units, rain sensors, etc.) to determine relevant information about the traffic scene, driver behavior, and driving duration based on factors such as the vehicle's location, driving route, weather conditions, and traffic flow. For example, the acquisition module 10 receives images of the road environment outside the vehicle 100 from the vehicle's panoramic visual perception system and then uses a trained machine learning model (e.g., an object classifier or convolutional neural network) to classify the road type (highway, urban road, or rural road). For another example, the acquisition module 10 can also receive precipitation information from the vehicle's rain sensor and traffic flow information for the current road section from the vehicle's positioning and navigation unit. Furthermore, the acquisition module 10 can use appropriate image recognition algorithms to determine the classification of other traffic participants in the vehicle's surroundings (e.g., the presence of vulnerable road users).

[0057] The providing module 20 is used to provide at least one reward interaction function based on the demotion information, and the reward interaction function is configured to encourage the driver to actively exit the autonomous mode of the vehicle and switch to the manual mode through interaction with the driver. To this end, the providing module 20 is connected to at least one display unit 30 of the vehicle 100, for example, to convey the reward effect by controlling the information push and screen change of the display unit 30. Here, the display unit 30 includes but is not limited to the vehicle-mounted imaging system of the vehicle 100, such as a head-up display (HUD: Heads-up-Display), an entertainment system display (HU: Head Unit), an instrument cluster (IC: Instrument Cluster), and a centralized vehicle-mounted display (CIVIC: Centralized In-Vehicle Integration Computer). In addition, the providing module 20 can also be connected to the voice output unit of the vehicle 100, so that the reward interaction effect can also be provided additionally through voice prompts.

[0058] It should be noted here that although the individual submodules 10, 20 of the device 1 Figure 1 Although shown as communication interfaces and connected to various sensors or actuators, it is also possible that these modules 10 , 20 are directly constructed as or include the above-mentioned on-board sensors and actuators.

[0059] Figure 2 A flow chart of a method for supporting manual driving of a driver of a vehicle according to an exemplary embodiment of the present invention is shown. Figure 2 In the embodiment shown, the method exemplarily includes steps S1-S2, and optionally includes step S2'. These steps can be performed using Figure 1 The device shown is implemented in the case of 1.

[0060] In step S1, degradation information is acquired during vehicle driving, where the degradation information reflects the driver's manual takeover of the driving task from the vehicle's autonomous mode. This includes both proactive takeover of the driving task from the vehicle's autonomous mode in the absence of a degradation reason, and reactive takeover of the driving task in response to a takeover request from the vehicle due to specific driving conditions or design operating domain limitations.

[0061] In a vehicle's autonomous mode, the driver does not need to continuously monitor the vehicle and the driving environment. Instead, the vehicle independently performs functions such as lane changes, adaptive cruise control, lane keeping, and automated driving. This allows the driver to focus on other tasks. However, in some levels of autonomous mode, the vehicle may prompt the driver to take over the driving task if necessary, meaning that the vehicle returns from autonomous mode to manual mode. In this case, the handover of driving tasks is mandatory, and therefore, in this context, it is also defined as "manual takeover of the driving task when there is a reason to take over" or "passive takeover of the driving task."

[0062] Contrary to the "takeover reason present" scenario defined above, it's also possible for the driver to spontaneously intervene in the vehicle's autonomous driving behavior, manually override the vehicle's automatically executed actions, or actively exit the vehicle's autonomous mode, without any predefined takeover scenario present and without any perceptible takeover command being output. For example, the driver's posture or behavior can be monitored using appropriate sensors to determine the driver's intention to switch modes. Furthermore, switching between different control modes can be identified based on vehicle driving data. Therefore, in this context, this is also defined as "manual takeover of the driving task without a takeover reason present."

[0063] In an optional embodiment, the vehicle's autonomous modes include at least a first autonomous mode and a second autonomous mode, with the second autonomous mode corresponding to a lower level of automated driving than the first autonomous mode. In this case, the downgrade information may also reflect, for example, a driver's switch from the first autonomous mode to the second autonomous mode. This includes both active driver-initiated switching from the first autonomous mode to the second autonomous mode in the absence of a downgrade reason and passive downgrades due to specific driving conditions or design operating domain limitations.

[0064] In step S2 , at least one reward interaction function is provided based on the degradation information, where the reward interaction function is configured to encourage the driver to actively exit the autonomous mode of the vehicle and switch to the manual mode through interaction with the driver.

[0065] Particularly advantageously, the reward results can be communicated directly to the user via at least one display in the vehicle or the user's mobile device. Furthermore, based on the driving degradation information, a gaming interface or virtual reality animation can be projected onto the vehicle's heads-up display, or the vehicle's ambient lighting can be automatically triggered to display effects.

[0066] In one specific example, reward points can be allocated to the driver based on the downgrade information, and the number of reward points can be visually and / or audibly displayed to the driver. As the user actively switches to manual mode, the number of points accumulated to date can be reported to the user in real time, or the number of points can be periodically displayed when a certain number of reward points is reached.

[0067] In another example, a virtual redemption service can be provided based on downgrade information. This virtual redemption service can automatically convert the downgrade information into virtual currency and store it in an account associated with the user's identity information. This virtual redemption service can also provide multiple redemption options: unlocking skill levels (different titles, which can be displayed on the vehicle screen in real time during driving), virtual gifts (car-themed screensavers, car-themed wallpapers), physical gifts (car interior decoration), and / or discounts on paid vehicle services (repair / maintenance / beauty).

[0068] In another example, the reward interaction function may also include a task collection consisting of multiple game tasks, and the collected demotion information may be associated with the completion progress of these game tasks. For example, as the user's manual driving time increases or as the number of times the user actively switches to manual mode increases, the completion progress of the game task is updated accordingly. For another example, the game content is provided in a virtual reality manner (for example, on a vehicle's head-up display). As manual driving progresses, the virtual props in the game task can be dynamically changed in combination with the actual driving scene. Therefore, the completion progress of one or more game tasks in the task collection can be obtained based on the demotion information, and the completion progress can be output to the driver in real time. Here, the specific forms of the game tasks may be, for example: level-breaking, challenge, and driving induction (parkour).

[0069] To further motivate users to hone their manual driving skills, the reward effect of this reward interaction feature is not a one-time benefit, but rather a continuous or repeatable one. For example, as the demotion information reflects an improvement in manual driving skills, the reward level pushed to the driver increases accordingly.

[0070] In an optional step S2', at least one additional reward interaction function is additionally provided based on the downgrade information. This additional reward interaction function is configured to, through interaction with the driver, encourage the driver to proactively exit the first autonomous mode and switch to the second autonomous mode, even when there is no reason for downgrade. For example, if the second autonomous mode is a Level 2 autonomous mode and the first autonomous mode is a Level 4 autonomous mode, when the system detects that the driver has proactively exited the Level 4 autonomous mode and switched to the Level 2 autonomous mode, reward points may be allocated to the user based on this information. However, compared to switching directly from the Level 4 autonomous mode to the manual mode, the number of reward points allocated to the user in the case of "merely downgrading the autonomous driving level" may be smaller, or the reward points may accrue more slowly. It is also possible that when the user passively takes over the driving task in response to a vehicle request, fewer reward points may be allocated, or the reward points may accrue more slowly, than when the user proactively takes over the driving task without a reason for taking over. This differentiation in the reward mechanism can further encourage users to spontaneously practice their manual driving skills.

[0071] Figure 3 In one exemplary embodiment, Figure 2 A flow chart of the steps of the method shown. Figure 3 In the embodiment shown, Figure 2 Step S1 in, for example, includes sub-steps S11-S17.

[0072] In step S11, the driver's behavior is monitored using sensors. For example, a camera located inside the vehicle cabin can capture an image of the driver and identify the position of the driver's hands or feet, sitting posture, gaze direction, and / or facial expressions. Furthermore, an onboard microphone can be used to receive voice messages from the driver. Additionally or alternatively, onboard sensors can be used to monitor the driver's actuation of specific vehicle components. For example, tactile sensors located on the steering wheel or touchscreen can be used to detect whether the driver is in contact with the steering wheel, the steering angle of the steering wheel, or whether a button indicating a mode switch is pressed. Furthermore, an accelerator pedal sensor or a brake pedal sensor can be used to detect the amount of operation of the accelerator or brake pedal by the driver.

[0073] In step S12, the driver's behavior is checked to see if it meets predefined criteria. For example, behaviors that could indicate a user's intent to take over or downgrade mode can be defined and categorized in advance. The detected user behavior is then compared with these predefined reference behaviors to determine if there is an intent to manually take over.

[0074] For example, manual driver takeover of the driving task from the vehicle's autonomous mode can be determined when the following predefined driver behaviors are detected:

[0075] - Detection that at least one of the driver's hands is holding the steering wheel;

[0076] - receiving a driver's voice command, gesture command, or touch command on a predefined key to activate manual mode; and / or

[0077] - Detecting that at least one of the driver's feet is on the accelerator pedal and / or brake pedal of the vehicle.

[0078] If no predefined behavior of the user is detected, the determination is continued in step S12.

[0079] If it is found in step S12 that one or more of the conditions listed above are met, a credibility check is performed on this result in the next step S13. As an example, a prompt message can be output to the driver of the vehicle to indicate that the vehicle is switching to manual mode or downgrading to autonomous mode. For example, the indicator light on the steering wheel can be triggered to light up, the color or brightness of the ambient light can be changed, or a voice prompt can be output. If a positive response from the driver to this prompt message is received, that is, if the driver confirms through action, gesture, touch operation, or voice, the check result in step S12 is confirmed to be credible, and therefore it can be confirmed in step S14 that the driver has manually taken over the driving task from the vehicle's autonomous mode. Conversely, if no positive response from the user to the prompt message is received or if an operation to reject the switch is received, it means that the user may have simply accidentally touched the steering wheel or a specific button. In this case, the process can return to step S12 and recheck whether the user's behavior meets the predefined behavior.

[0080] When it is confirmed in step S14 that the user has manually taken over the driving task from the autonomous mode, data related to the user's manual driving operation, the operation of the vehicle, and the road conditions outside the vehicle may be collected in step S15.

[0081] As the vehicle travels, it may be checked in step S16 whether the current driving mode of the vehicle has changed again, for example, whether the vehicle has returned from the manual mode to the autonomous mode.

[0082] If it is found that the driving task of the vehicle is taken over by the vehicle again, the collection of degradation information is stopped in step S17.

[0083] Figure 4 In another exemplary embodiment, Figure 2 A flow chart of the steps of the method shown. Figure 4 In the embodiment shown, Figure 2 Step S1 in, for example, includes sub-steps S11-S17. Figure 4 The degradation information collection process in Figure 3 The embodiment shown is carried out in a similar manner, and the following only focuses on Figure 4 and Figure 3 The difference between:

[0084] refer to Figure 4 In step S13', the driver's manual takeover behavior is classified. For example, whether the detected manual takeover behavior is passively initiated or actively initiated can be checked based on whether the driver's affirmative response to the mode switch is received.

[0085] If an affirmative response to the mode switch prompt is received from the driver, then in step S141 it is confirmed that the driver has proactively taken over the driving task from the autonomous mode of the vehicle in the absence of a takeover reason.

[0086] If no affirmative response to the mode switching prompt is received from the driver or if an operation of the driver rejecting the mode switching is received, it is confirmed in step S142 that the driver passively takes over the driving task in response to the takeover reason.

[0087] When the cause of the driver manually taking over the driving task is known, the following steps S15-S17 can be used to Figure 3 The information during manual driving is collected in a similar manner. However, the reason for manually taking over the driving task can be stored so as to be used in the following step S2 to control the specific triggering mode of the reward interaction function.

[0088] Figure 5 In one exemplary embodiment, Figure 2 A flow chart of another method step of the method shown. Figure 5 In the embodiment shown, Figure 2 Step S2 in, for example, includes sub-steps S21-S26.

[0089] In step S21, information related to the traffic scene during the driver's manual driving of the vehicle is extracted from the degradation information acquired in step S1. For example, the following condition information c is extracted from the degradation information that has been collected: i , i=1…n:

[0090] - Weather conditions c1, such as the amount of precipitation on the road the vehicle is traveling on using a vehicle rain sensor or a wiper activation signal, and the light intensity, visibility, and pollution level outside the vehicle using an optical sensor;

[0091] - road category c2, which can be read from a digital map in conjunction with the vehicle's GPS positioning information, or can also be determined based on image recognition technology to determine whether the current road section is a city road, a motorway or a country road;

[0092] The presence or number c3 of vulnerable road users, which can also be determined, for example, using image recognition technology;

[0093] Traffic flow c4, which can be obtained, for example, by reading the real-time updated road condition information in the positioning and navigation unit, or by using image recognition technology;

[0094] The time of day c5 , which can be read, for example, from the time unit of the vehicle.

[0095] As an example, the time correlation of the above condition information can be considered. In other words, each condition information c can be continuously updated as time changes. i (t). For example, the currently stored condition information c can be updated every five seconds. i (t) Perform an update.

[0096] In step S22, the extracted multiple condition information c i (t) is provided as input to the trained machine learning model, and then the difficulty level of the traffic scene d(t,c i (t)), where t represents time.

[0097] As an example, the machine learning model has been pre-trained with a large amount of sample data and is trained to directly use the difficulty coefficient d(t,c i The difficulty level of the traffic scene is output in the form of (t). The higher the difficulty level of the traffic scene, the higher the difficulty coefficient d(t,c i For example, when a driver manually drives a vehicle on a highway with light traffic and wide lanes, the difficulty coefficient d(t,c i (t)) is small. When the driver manually drives through a densely trafficked intersection, and if the weather conditions are bad (for example, it is raining), the difficulty coefficient d(t,c) corresponding to this traffic scenario is i (t)) is large. Of course, this difficulty coefficient is not static, but is constantly updated based on the input data provided to the machine learning model.

[0098] In this embodiment, steps S21' and S22' are further executed in parallel with steps S21-S22 or alternately with each other. In step S21', information on the driver's behavior is extracted from the acquired degradation information.

[0099] For example, the following driver behaviors can be identified directly from cabin images taken with autonomous mode disengaged:

[0100] - Driver's facial expression (stressed / relaxed);

[0101] -Whether the driver's line of sight remains on the road;

[0102] -Whether the driver was wearing a seat belt during manual driving;

[0103] - Whether the driver is hesitant when performing manual steering / lane changes.

[0104] Additionally, the following vehicle driving data can be extracted when autonomous mode is detected to be disengaged:

[0105] - The vehicle's trajectory;

[0106] -Vehicle speed and acceleration;

[0107] -Number of direction changes and lane changes of the vehicle;

[0108] - the steering angle of the vehicle's steering wheel;

[0109] -The number of times / amount of operation of the vehicle's accelerator pedal or brake pedal.

[0110] In step S22', the driver's performance during manual driving can be evaluated and a performance score p(t) calculated therefrom. For example, a deviation between the driver's manual driving behavior and the expected driving behavior can be determined. If the deviation exceeds an allowable amount, a negative or low score p(t) is assigned to the driver's performance. If the deviation does not exceed the allowable amount, a positive or high score p(t) is assigned to the driver's performance.

[0111] For example, the behavioral performance score p(t) may be obtained based on at least one of the following:

[0112] - During the period when the driver is manually driving the vehicle, the greater the degree of agreement between the vehicle's actual driving trajectory and the predefined reference driving trajectory, the higher the behavior performance score p(t);

[0113] - During the period when the driver is manually driving the vehicle, the smaller the number and / or degree of deviation of the vehicle's driving speed from the predefined reference speed interval, the higher the behavior performance score p(t);

[0114] - the fewer times the driver accelerates and / or brakes beyond a preset magnitude threshold during manual driving of the vehicle by the driver, the higher the behavioral performance score p(t); and / or

[0115] -The longer the vehicle's relationship with other surrounding traffic objects continues to meet predefined requirements during the driver's manual driving of the vehicle, the higher the behavior performance score p(t).

[0116] In step S23, the driver is assigned bonus points in relation to the difficulty level of the traffic scene during which the driver manually drives the vehicle, the driver's performance during the manual driving of the vehicle, and the duration of the driver's manual driving of the vehicle. In this step, for example, the difficulty coefficient d(t,c i (t)) and the driver's behavior performance score p(t) are summarized, and for each time increment, the incremental reward points are calculated weightedly in terms of the difficulty level of the traffic scene and the driver's behavior performance, and then the incremental reward points of each time increment are accumulated to form the total reward points so far.

[0117] In a specific example, the reward points may be calculated according to the following equation:

[0118] n(t)=n(t–Δt)+d(t,c i (t))+p(t) (1)

[0119] Where t represents the manual driving time up to now, Δt represents the time increment relative to the last summary of reward points, n(t–Δt) represents the number of reward points accumulated when the reward points were last summarized, n(t) represents the number of reward points accumulated when the reward points are summarized at the current moment, and d(t,c i Where p(t) represents the difficulty coefficient of the traffic scenario for the time increment Δt, and p(t) represents the driver's performance score for the time increment Δt. Other methods for calculating the reward points or other weighting ratios are also conceivable, and all fall within the scope of the present invention.

[0120] In another example, different calculation methods of reward points can be selected according to whether the demotion information reflects the existence of a takeover reason or a demotion reason. For example, if the demotion information reflects the absence of a takeover reason or a demotion reason, the reward points n(t) can be directly calculated according to the above equation (1). If the demotion information reflects the existence of a takeover reason or a demotion reason, for example, the items d(t, c) representing the difficulty coefficient can be respectively calculated. i A coefficient q is added before the term p(t)) and the driver's performance term p(t), where q is, for example, a percentage between 0 and 100%, and then the bonus points are calculated according to the following equation:

[0121] n(t)=n(t–Δt)+q*d(t,c i (t))+q*p(t) (2)

[0122] The value of the coefficient q can be dynamically adjusted according to the degree of reduction in the autonomous driving level and the necessity of the takeover reason.

[0123] Next, in step S24, the currently accumulated reward points n(t) are compared with the point threshold m i Compare, where i = 1, 2…n and represents different skill levels, and then check whether n(t) exceeds a specific integral threshold m i Here, the reward interaction function includes, for example, multiple skill levels (such as basic driver, advanced driver, professional driver, master driver, top master driver), and different skill levels are bound to different point thresholds.

[0124] For example, if it is found that the currently accumulated reward points n(t) just exceed the points threshold m2 corresponding to "advanced driver", a prompt is output to the driver in step S25 that the skill level "advanced driver" bound to the points threshold m2 is unlocked.

[0125] If it is found that the currently accumulated bonus points n(t) have not exceeded the points threshold m2, and the skill level "Basic Driver" is in an unlocked state, the difference m2-n(t) between the bonus points n(t) and the points threshold m2 is output to the driver in step S26. In this way, the driver can clearly understand how many bonus points he needs to obtain through manual driving operations if he wants to upgrade from "Basic Driver" to "Advanced Driver".

[0126] Figure 6a 、 Figure 6b 、 Figure 6c 、 Figure 6d A schematic diagram illustrating an interactive interface of a reward interaction function according to an exemplary embodiment of the present invention is shown.

[0127] exist Figure 6a In the illustrated scenario, reward points 51 are allocated to the driver based on the user's demotion information, and these reward points 51 are displayed in real time on the vehicle's display unit 30. While the vehicle is being manually driven, these reward points 51 are updated, for example, at unit time increments (e.g., every 10 seconds). This allows the user to understand the correlation between their current driving behavior and the reward points 51 in real time, and accurately understand changes in the reward points 51. Furthermore, the number of reward points 51 displayed to the user may also change depending on whether the vehicle is being driven safely or the traffic scenario the user is in.

[0128] exist Figure 6bIn the illustrated scenario, instead of directly displaying the number of reward points in real time, the vehicle's display unit 30 displays the unlocked skill level 52 corresponding to the number of reward points. In this embodiment, the user's accumulated reward points through actively undertaking driving tasks have exceeded a certain threshold, so a prompt 52 indicating that they have unlocked "Master Driver" is displayed. A prompt 53 indicating the gap between the current number of reward points and the next skill level is also displayed. In this example, the unlocked skill level is indicated by solid blocks, while the unlocked skill level is indicated by dashed blocks.

[0129] exist Figure 6c In the illustrated scenario, multiple reward redemption options 54 and the required number of reward points appear on the vehicle's display unit 30. These options include, for example, game props, vehicle-themed wallpapers, vehicle-themed screensavers, and maintenance discounts. The driver can provide a redemption instruction via voice or touch control, which executes the point redemption operation and sends a notification to the driver indicating that the redemption is successful.

[0130] exist Figure 6d In the illustrated scenario, in response to the user exiting autonomous mode and entering manual mode, a level-breaking game mode automatically begins. Here, virtual reward coins 55 and virtual diamonds, which must be collected during the game mission, are projected onto the vehicle's front windshield and displayed superimposed on the road environment visible through the windshield. These virtual rewards are, for example, distributed along the vehicle's navigation route. When the driver drives along the navigation route, they can earn these virtual rewards 55; if the vehicle deviates from the navigation route, they cannot earn virtual rewards. Furthermore, the type and distribution of virtual rewards 55 change as the traffic scene changes, thereby enriching the user's manual driving experience.

[0131] Although specific embodiments of the present invention are described in detail herein, they are provided for illustrative purposes only and should not be considered to limit the scope of the present invention. Various substitutions, changes, and modifications may be conceived without departing from the spirit and scope of the present invention.

Claims

1. A user interaction method for a vehicle, the vehicle (100) being driven at least partially autonomously, the user interaction method comprising the following steps: S1: Detecting degradation information during driving of the vehicle (100), wherein the degradation information reflects that a driver manually takes over a driving task from an autonomous mode of the vehicle (100), and the degradation information includes the driver's behavior during manual driving of the vehicle (100); S2: providing at least one user interaction strategy based on the detected degradation information, wherein the user interaction strategy includes outputting a number of reward points, The driver's performance during manual driving is evaluated, and a performance score p(t) is calculated based on the performance score p(t). The reward points are calculated based on the performance score p(t). The deviation between the driver's manual driving behavior and the expected driving behavior is determined, and if the deviation exceeds an allowed amount, a negative score or a small score p(t) is assigned to the driver's behavior performance; if the deviation does not exceed the allowed amount, a positive score or a large score p(t) is assigned to the driver's behavior performance. Identifying changes in the driver's manual driving experience and / or manual driving proficiency based on the degradation information; and / or dynamically updating the triggering method of the user interaction strategy and / or the interaction method with the driver in response to identifying the changes in the manual driving experience and / or manual driving proficiency.

2. The user interaction method according to claim 1, wherein: The degradation information reflects that the driver manually takes over the driving task from the autonomous mode of the vehicle (100) when there is no reason for taking over.

3. The user interaction method according to claim 1 or 2, wherein: The step S2 comprises: allocating reward points to the driver based on the downgrade information (51); and The number of bonus points (51) is output to the driver visually and / or audibly.

4. The user interaction method according to claim 3, wherein: The user interaction strategy includes multiple skill levels, and different skill levels are bound to different score thresholds, wherein: - when the bonus points (51) allocated to the driver exceed a determined point threshold, outputting a prompt (52) to the driver that the skill level bound to the point threshold is unlocked; and / or - when the bonus points (51) allocated to the driver do not exceed a determined point threshold, outputting a difference (53) between the bonus points (51) and the determined point threshold to the driver.

5. The user interaction method according to claim 3, wherein: In step S2, reward points (51) are allocated to the driver in relation to the difficulty level of the traffic scene during which the driver manually drives the vehicle (100) and the duration of the driver manually driving the vehicle (100).

6. The user interaction method according to claim 3, wherein: In step S2, while the driver is manually driving the vehicle (100), incremental reward points are weightedly calculated for each time increment in terms of the difficulty level of the traffic scene and the driver's behavior performance, and the incremental reward points of each time increment are accumulated to form the reward points (51).

7. The user interaction method according to any one of claims 1, 2, 4, 5 and 6, wherein: The behavior of the driver during manual driving of the vehicle (100) is obtained based on at least one of the following: - the degree of agreement between the actual driving trajectory of the vehicle (100) and the predefined reference driving trajectory during manual driving of the vehicle (100) by the driver; - the number and / or extent to which the driving speed of the vehicle (100) deviates from a predefined reference speed interval during manual driving of the vehicle (100) by the driver; - the number of times the driver accelerates and / or brakes exceeding a preset magnitude threshold during manual driving of the vehicle (100); and / or - The relationship of the vehicle (100) to other surrounding traffic objects during manual driving of the vehicle (100) by the driver.

8. The user interaction method according to any one of claims 1, 2, 4, 5 and 6, wherein: The degradation information includes a difficulty level of a traffic scene during manual driving of the vehicle (100) by a driver, wherein the difficulty level of the traffic scene is determined by using a trained machine learning model based on weather conditions, road type, traffic flow and / or the presence of vulnerable road users of the traffic scene.

9. The user interaction method according to any one of claims 1, 2, 4, 5 and 6, wherein: The autonomous mode of the vehicle (100) includes at least a first autonomous mode and a second autonomous mode, wherein the degradation information further reflects that: the driver switches from the first autonomous mode of the vehicle (100) to the second autonomous mode, and the autonomous driving level corresponding to the second autonomous mode is lower than the autonomous driving level corresponding to the first autonomous mode; In step S2 , the user interaction strategy is further configured to encourage the driver to actively exit the first autonomous mode and switch to the second autonomous mode through interaction with the driver.

10. The user interaction method according to claim 9, wherein: In step S2, the reward level determined for the first autonomous mode of the vehicle (100) is lower than the reward level determined for the second autonomous mode of the vehicle (100), and the reward level determined for the second autonomous mode of the vehicle (100) is lower than the reward level determined for the manual mode of the vehicle (100).

11. The user interaction method according to any one of claims 1, 2, 4, 5, 6 and 10, wherein: The user interaction strategy includes a virtual redemption service configured to output the following redemption options (54) to the driver based on the demotion information: unlocking of skill levels, virtual gifts, physical gifts and / or paid discounts for vehicle (100) services.

12. The user interaction method according to any one of claims 1, 2, 4, 5, 6 and 10, wherein: The user interaction strategy includes a task collection consisting of multiple game tasks. In step S2, the completion progress of one or more game tasks in the task collection is obtained based on the demotion information, and the completion progress is output to the driver.

13. The user interaction method according to any one of claims 1, 2, 4, 5, 6 and 10, wherein: In said step S1, it is determined that the driver manually takes over the driving task from the autonomous mode of the vehicle (100) when the following predefined behavior of the driver is detected: Detecting that at least one of the driver's hands is holding the steering wheel; Receiving a driver's voice command, gesture command, or touch command on a predefined button to activate manual mode; and / or It is detected that at least one foot of the driver is stepped on an accelerator pedal and / or a brake pedal of the vehicle (100).

14. The user interaction method according to any one of claims 1, 2, 4, 5, 6 and 10, wherein: Said step S2 further comprises: in response to detecting a predefined behavior of the driver, outputting a prompt message to the driver of the vehicle (100) to switch the vehicle (100) to the manual mode, wherein the driver's manual taking over of the driving task from the autonomous mode of the vehicle (100) is confirmed only when a positive response to the prompt message is received from the driver.

15. The user interaction method according to claim 9, wherein: The downgrade information also reflects that, in the absence of a downgrade reason, the driver switches from the first autonomous mode of the vehicle (100) to the second autonomous mode, and the autonomous driving level corresponding to the second autonomous mode is lower than the autonomous driving level corresponding to the first autonomous mode.

16. A device (1) for supporting manual driving of a driver of a vehicle (100), the device being configured to execute the user interaction method according to any one of claims 1 to 15, the device (1) comprising: An acquisition module (10) is configured to acquire degradation information during driving of the vehicle (100), wherein the degradation information reflects that a driver manually takes over the driving task from the autonomous mode of the vehicle (100); as well as A module (20) is provided, which is configured to provide at least one user interaction strategy based on the degradation information.

17. A vehicle (100) that travels at least partially autonomously, the vehicle (100) comprising a device (1) according to claim 16. 18 . A machine-readable storage medium having a computer program stored thereon, wherein the computer program is configured to execute the user interaction method according to claim 1 when the computer is executed.

Images