Method and system for situation-dependent handling of a driver's interaction with an infotainment system

Abstract

The invention relates to a method and a system for situation-dependent handling of an interaction between a driver (10) and an infotainment system of a moving vehicle, in which content that is distracting for the driver (10) is detected and, as a result, a handover to a passenger (12) is to take place. For this purpose, it is checked whether the passenger (12) is already interacting with the infotainment system and the handover to the passenger is designed accordingly.

Description

[0001] The invention relates to a method and a system for situation-dependent handling of a driver's interaction with an infotainment system of a moving vehicle, in which content that is distracting to the driver is detected and, as a result, a handover to a passenger is to take place. For this purpose, it is checked whether the passenger is already interacting with the infotainment system and the handover to the passenger is adjusted accordingly.

[0002] Ideally, a driver should be distracted as little as possible while driving. Nevertheless, situations arise where interaction with the vehicle's infotainment system becomes necessary, for example, when a passenger wants to connect their mobile device, when a route needs to be searched for and started, and so on. Often, this input is made on the vehicle's center display, where the driver can see the input, thus distracting them from safe driving. Furthermore, input is often ergonomically inconvenient for a passenger, as the center display is usually angled towards the driver.

[0003] To prevent the driver from making the necessary entries or observing these entries, various solutions have been implemented. In most cases, input into the infotainment system is blocked when the vehicle is in motion. Input is then only possible for the front passenger or another passenger for whom the infotainment system is unlocked.

[0004] To detect the presence of a passenger and unlock the input for him, for example in GB 2 583 890 A1 an input option is disclosed which detects the presence of the passenger, for example by means of a weight sensor, or is arranged so far away from the driver that only the passenger can operate it.

[0005] An alternative approach is shown in the JP 2007 203 987 A2, which features a dual display where different content can be shown to the driver and the front passenger, and input for the navigation system is only possible from the front passenger.

[0006] However, the known solutions do not take into account whether such an input by the driver is actually distracting and / or whether a passenger, and in particular the front passenger, is already interacting with the infotainment system in another way and whether they are willing or able to make such an input on behalf of the driver.

[0007] The purpose of the invention is therefore to handle interaction with the infotainment system depending on the situation.

[0008] The object of the invention is achieved by a method according to claim 1 and a system according to claim 10. Further preferred embodiments of the invention result from the remaining features mentioned in the dependent claims.

[0009] The method according to the invention serves for the situation-dependent handling of an interaction with an infotainment system in a vehicle during driving operation, wherein the infotainment system can be used by both a driver and at least one of the passengers, and is designed with a display and input system with at least one display and input device, and comprises at least the following steps: (S1) Detection of content that may distract the driver, (S2) Detection of other use of the infotainment system by a selected passenger, (S3) if other use is detected: query the driver to ask whether the interaction should be transferred to the selected passenger, (S4) if the query is confirmed: handover of the interaction to the selected passenger, and (S5) if the query is denied: Abort the query and block the interaction for the driver.

[0010] The procedure aims to ensure that, where possible, distracting content for the driver is transferred to a passenger in the vehicle, enabling the passenger to interact with it. Interaction includes the viewer perceiving the displayed content and, if necessary, entering information, making selections, and confirming actions within a user dialog. In the simplest case, this could involve adjusting the vehicle's interior temperature, but it could also include searching for, selecting, and starting a route for vehicle navigation.

[0011] Detecting whether there is a passenger or passengers in a vehicle besides the driver, and where they are seated, is a well-known procedure and will therefore not be described further. This procedure is executed while the vehicle is in motion. It can also be applied at the start of a journey if the vehicle begins moving and the driver interacts with the infotainment system before or during this process. The detection of active driving is also well-known from the prior art and will therefore not be described. A passenger is any vehicle occupant who is not the driver, and in particular, a front-seat passenger.

[0012] The vehicle features an infotainment system that can be operated by the driver and at least one passenger. Ease of use depends on the spatial arrangement of the infotainment system and its input interfaces, as well as the seating position of the passengers in the vehicle. However, there may also be a functional limitation, such as allowing only the driver and front passenger to operate the infotainment system.

[0013] The infotainment system includes a display and input system, which in turn comprises at least one display and input device. In the simplest case, a display and input device is a touch-sensitive screen, in other words, a touchscreen. However, a display and input device can also be a screen (display) with separate controls. Any configuration is possible, allowing both the display of content and interaction in the sense of operation. At least one of the display and input devices constitutes the display and input system; therefore, it can also be configured with two or more display and input devices.The display and input system can also include other input elements that trigger an interaction with the vehicle or vehicle functions, which are then visualized on a display device, such as a temperature control for seat heating.

[0014] In a first step, content that distracts the driver is detected. This can be done continuously in the background based on criteria that will be explained in more detail with regard to optional configurations of the procedure. Distracting content can include the playback of films, images, combined media content, and the like, but also searching for, selecting, and, if necessary, adjusting a route using the infotainment system, or changing various settings in the vehicle. Distracting content can therefore be passive, such as media consumption, or active, requiring interaction with the infotainment system. Such media can include, for example, videos or images containing important information about the journey or the destination, which may have been transmitted or provided by people at the destination.

[0015] A distinction can be made between (active) distracting content that requires a single, brief interaction, such as increasing the temperature in the vehicle, and repeated interaction that also involves considering choices and making decisions. The latter is generally considered distracting content.

[0016] Only after distracting content has been detected is a second step performed to check whether a selected passenger is using the infotainment system in another way. Other use could include, for example, watching a film on a second display or input device, or interacting with the infotainment system itself, i.e., operating it.

[0017] A selected passenger is the passenger for whom this check should be performed first. In the simplest case, this could be the front passenger. However, a user-defined pre-selection can be made, for example, if a child is in the front passenger seat and another adult is in the back seat, where the latter has their own display and input device with which they can interact. If there are multiple passengers in the vehicle, a query order can also be defined, as will be explained elsewhere. The selection can also be automated, for example, by differentiating between adults and children based on image data, or between known and unknown vehicle occupants.

[0018] If the system detects that the selected passenger is using the infotainment system in a different way, the driver is prompted to decide whether to transfer the interaction they intend to perform to the selected passenger (step 3). This prompt can be displayed as text with multiple-choice options, graphics, and / or voice output. This prompt is useful for several reasons: if the selected passenger is consuming media, such as watching a film, and falls asleep, they cannot take over the interaction. The driver may also not want to disturb the selected passenger while they are watching a film. The selected passenger may be completely or temporarily unable to perform the interaction because, for example, they are unfamiliar with the area, ill, intoxicated, or a child who is too young to perform the interaction.Physical impairments can also make taking over the interaction impossible. Furthermore, the selected passenger may need to use the infotainment system for other purposes, such as a search using a search application, adjusting the interior lighting and / or climate control, or route planning, and this should not be interrupted. At this point, the driver can decide whether transferring the interaction to the selected passenger is possible, advisable, and / or practical.

[0019] If the query in step 3 is answered in the affirmative, the fourth step involves transferring the interaction to the selected passenger, in particular to a display and input device assigned to him or accessible to him, so that he can carry out the necessary interactions.

[0020] If the query in step 3 is answered with "no," the query is canceled and the interaction is blocked for the driver (step 5), preventing further distraction while driving. The interaction or distracting content initiated by the driver is preferably not displayed further, preventing them from continuing it. Optionally, a message can be displayed informing the driver that this interaction or content cannot be accessed or consumed while driving.

[0021] The proposed method makes it possible, depending on the situation, to transfer a driver's interaction with the vehicle's infotainment system to a passenger while driving, provided this interaction is recognized as distracting. This reduces the risk of driving errors due to inattention and distraction. The interaction is then performed by the selected passenger on the driver's behalf.

[0022] In a first embodiment of the method according to the invention, in step S2 it is checked whether one of the passengers does not use the infotainment system or uses it less, and this passenger is designated as the selected passenger.

[0023] As previously explained, when multiple passengers are traveling together, a pre-selection or automated selection can be made among the passengers to determine the selected passenger. In this configuration, this selection will now be based on whether or not the infotainment system is being used by other passengers.

[0024] If one of the passengers does not use the infotainment system, or uses it less actively than other passengers, then it can be assumed that transferring an interaction to them is less disruptive than transferring it to a passenger who, for example, is watching a film or actively entering data into the infotainment system. This passenger then becomes the selected passenger to whom the interaction can be transferred.

[0025] In a second embodiment of the procedure, step S3 is omitted if no other use by the selected passenger is detected. Thus, if the check to see if the selected passenger is using the infotainment system in any other way reveals that they are not, the prompt to the driver asking whether the interaction should be transferred can be skipped, and the interaction can be transferred directly to the selected passenger. This avoids an unnecessary prompt to the driver and reduces the number of inputs they need to make regarding the transfer.

[0026] This configuration is particularly suitable in combination with the previous one, where the selected passenger is the one who does not use the infotainment system or uses it less than other passengers, and especially in the case that he does not actually do so.

[0027] A particularly preferred embodiment of the method according to the invention is if, in the course of step 4, as step 4a, a query is made to the selected passenger as to whether he wants to take over the interaction, wherein • if he agrees, the interaction will be handed over to him and • If rejected, the query is either sent back to the driver or the query is cancelled and the interaction is blocked for the driver.

[0028] This configuration can be implemented both when the selected passenger is using the infotainment system for other purposes and when they are not. This allows for verification of whether the selected passenger is attentive enough to notice the query, and also whether they are willing and able to do so.

[0029] If the driver has decided in step 3 that the interaction should be transferred to the selected passenger, the selected passenger is asked (step 4a) whether they wish to take over this interaction before the actual transfer (step 4). If they agree, the transfer takes place as described previously. This query to the selected passenger is carried out in the same way if the driver's prior query is omitted because the infotainment system is not in use.

[0030] If the selected passenger declines, the query can be aborted and the driver's interaction with the infotainment system can be blocked, as explained in step 5. Alternatively, the response can be sent back to the driver.

[0031] When the query is sent back to the driver, an optional prompt can be triggered to transfer the interaction to another passenger. The driver is then asked whether the interaction should be transferred to a different passenger than the one previously questioned. If they confirm, the query can be sent directly or with a further prompt to take over the interaction with the other passenger. This ensures that the interaction is completed instead of being immediately aborted.

[0032] If the requested passenger does not respond to the question of whether they can or want to take over the interaction, the system can be set to automatically return the request or cancel it after a predefined period. This can prevent unnecessary waiting times or unclear situations, for example, if the selected passenger is sitting behind the driver and cannot be identified.

[0033] The detection of distracting content can be achieved according to a further modification of the procedure. • based on a categorization and / or • by means of recording and evaluating displayed content and / or interactions with the infotainment system.

[0034] Various uses of a vehicle's infotainment system can be pre-categorized to determine whether they are considered distracting. This categorization can extend to applications the driver can launch or use, as well as to file or media types. An application might be the vehicle's route planner or a music app where playback selections are made; a media type could be a video or images. However, this categorization requires that the application or media type is known and assigned to the category. When it is launched, the categorization immediately identifies it as distracting content and triggers the subsequent steps of the process.

[0035] Alternatively or additionally, the classification as distracting content can be achieved by recording and evaluating displayed content and / or interactions with the infotainment system. This involves recording and assessing the effect of an application or media type. This approach is therefore advantageous for unknown and thus uncategorized applications, or for verifying an existing categorization.

[0036] For example, the number of image changes on a display and input device can be recorded within a given time period. This allows for the detection, or at least an estimation, of whether videos are being played or whether an application with many, potentially rapidly changing displays is distracting the driver. By recording and evaluating the frequency of inputs via the display and input device, it is possible to determine, or confirm, whether content is being played or whether the driver is actively interacting with the infotainment system, i.e., making decisions and entering data. This could then indicate potentially persistent menu navigation.

[0037] Here, too, a distinction can be made between a short interaction with few inputs, such as temperature control, and a more complex one that can actually be distracting.

[0038] To prevent or at least restrict the viewing of the planned or blocked interaction of the driver with the infotainment system, a visual shielding of the driver's interaction can be implemented using step 4 or the optional step 4a.

[0039] In the simplest case, this can be achieved by overlaying the content with a notification or displaying such a notification instead of the content itself, thus preventing access while driving. Overlaying or covering the content with a graphic, or blurring the content, can also achieve this effect. The goal is to prevent the driver from recognizing the planned or blocked interaction, thereby rendering observation unnecessary.

[0040] To ensure that the driver does not or cannot monitor the interaction of the selected passenger with the infotainment system, this can be hidden from the driver by means of a switchable filter.

[0041] Such a switchable filter can be a coating or film on the display and input device, activated by the selected passenger. This coating, activated by a switching action (i.e., current), creates a privacy effect, so that the content of the interaction is only visible when viewed directly from a predefined angle, but not from the driver's position. This privacy filter can be activated as needed to prevent the driver from being distracted by the selected passenger's interaction with the infotainment system. Outside of this interaction, the filter can revert to its previous state.

[0042] In this way, it is advantageous to enable a single, shared display and input device for use by a passenger as a selected passenger.

[0043] As stated above, the display and input system should have at least one display and input device. Therefore, in one configuration, it can be formed with at least one divisible display and input device and / or at least two display and input devices.

[0044] A divisible display and input device can include a display that can be divided into two separate areas, side by side or one above the other, with one area displaying content for the driver and the other for the front passenger. This is achieved by controlling the display and input device. The display and input device can also be shaped so that parts of it face the front passenger to mitigate the ergonomic issues described above.

[0045] A subdivision can also be achieved through a direction-dependent display, as is done with so-called dual displays, where two different contents are provided simultaneously, which are visible or not depending on the viewing angle of the display.

[0046] This division will be particularly useful for the use of the display and input device by the driver and his passenger.

[0047] The display and input system can also be configured with at least two display and input devices, one of which is assigned to the driver and can be operated by him, and at least one further device for the passenger or passengers, for example, one for the front passenger or at least one in the rear of the vehicle. These can each be controlled individually within the procedure.

[0048] To implement the method according to the invention, a system is provided which is designed and configured to implement precisely this method. For this purpose, the system includes a control unit or interacts with one, which is communicatively connected to the display and input system to capture inputs there, but also executes applications (apps) or plays media used by the driver and / or passengers. Thus, information on infotainment usage is collected in the control unit and can be evaluated to serve as the starting point for the method according to the invention.

[0049] The procedure itself is stored as a sequence of commands or program code on a storage device of the control unit or on an external storage device to which the control unit has access; the procedure is implemented upon execution of this sequence. The procedure can also be implemented by an algorithm.

[0050] The system can be provided in a vehicle or implemented by the vehicle itself.

[0051] The inventive method and associated system enable the detection and prevention of driver distraction in a vehicle, depending on the situation, by transferring the distracting interaction from the driver to a selected passenger, who then carries it out and completes it. Depending on the specific (optional) implementation of the method, a planned delegation can be confirmed in advance by the driver and / or rejected by the selected user. This reduces driving errors caused by driver inattention and lowers the risk of accidents.

[0052] Unless otherwise stated in individual cases, the various embodiments of the invention mentioned in this application can be advantageously combined with one another.

[0053] The invention is explained below using exemplary embodiments with reference to the accompanying drawings. These show: Fig. 1 an exemplary implementation of the method according to the invention, and Fig. 2. The use of switchable filtering in a passenger display and input device.

[0054] Fig. Figure 1 shows an example of how the method according to the invention can be implemented. It is assumed that the vehicle is already moving, but it can also be executed at the beginning of a journey while the driver 10 is still interacting with the infotainment system. The method is executed by a vehicle-integrated control unit with a storage device containing instructions for the method.

[0055] In Fig. Figure 1 represents a display and input system 14 for each individual process step, with a display and input device 16 in the form of a dual display. The left side of the display is assigned to the driver 10, and the right side to the passenger 12. This display is touch-sensitive and thus allows input. This arrangement serves only as a simplified illustration. Two separate displays can also be used.

[0056] In step S1, the system first detects whether the display and input device 16 (left side) contains content that could be distracting for the driver 10. This is demonstrated here using a prior categorization of the applications available via the infotainment system.

[0057] If this is the case (reference symbol Y), step S2 follows, in which it is detected whether the selected passenger 12, in this case the front passenger, is using the infotainment system for any other purpose. If there were other passengers in the vehicle, a preselection for an assignment sequence could optionally be made, or the assignment could be based on the intensity of infotainment system usage.

[0058] If this is the case (reference symbol Y), the one in Fig. The right branch of the procedure is followed. Thus, in step S3, a query is provided on the display assigned to the driver (10) asking whether the interaction with the infotainment system should be transferred to the passenger (12). The passenger is currently watching a video (V).

[0059] If the driver 10 confirms this (reference sign Y), the video V on the display assigned to the passenger 12 is overlaid with a prompt (step 4a) asking whether the driver 10 wishes to transmit an interaction and whether the passenger 12 wishes to accept this. Simultaneously, a message appears on the driver's display (step S7) stating that no input can be made at this point while driving and, optionally, that the prompt to the passenger 12 is running. In this exemplary embodiment, a cancellation option is provided for the driver 10, which, by selecting it in step S9, allows them to cancel or terminate the prompt.

[0060] If the passenger 12 confirms the acceptance of the interaction (reference sign Y), this is transferred to their display (step S4) and the driver 10 sees content S1' on their display, which may include a locked view of their last interaction or a message indicating that input is not possible while driving. Content S1' can also be the start screen of the infotainment menu.

[0061] If passenger 12 declines to accept the interaction (reference sign N), the query is sent back to driver 10. The driver can then forward the query to another passenger, if available, or request one (indicated only by reference sign S8), or abort the process.

[0062] If driver 10 does not want to transfer the interaction to passenger 12 (reference sign N after step S3) or aborts the procedure according to the previous paragraph, the query is aborted and the interaction is blocked in step S5. As a result, the content S1' can be displayed, as explained previously. The same result occurs if the query to passenger 12 is aborted in step S9.

[0063] If the front passenger 12 does not use the infotainment system (reference sign N after step S2), the procedure follows the left branch in Fig. 1. Then, without querying the driver 10 (i.e., without step S3), a query is sent to the passenger 12 asking if they wish to take over the interaction (step S4a). If yes, this is transmitted to their display (step S4), and the driver 10 is shown content S1', as already explained. If they decline (reference N), a message is sent back to the driver (step S6), who can then cancel the process in step S9. The passenger 12 receives no further notification.

[0064] During the query to the passenger in step S4a, a message is simultaneously displayed on the driver's display (step S7), indicating that the driver cannot make any inputs at this point while driving and optionally that the query to the passenger (step S2) is in progress. The query can also be canceled or terminated here (step S9).

[0065] If the procedure is terminated in step S9, the alternative content S1' will be displayed again.

[0066] If the interaction is transferred to the passenger 12, the driver 10 should be prevented from looking at it. Fig. Figure 2 shows the effect of a switchable filter 18 on a display 16 of the passenger. The display and control system 14 is in Fig. 2 of Fig. 1 different.

[0067] Once the interaction has been transferred to the passenger 12, the switchable filter 18 is simultaneously activated, so that the driver 10, depending on their size and seating position, may not be able to see the passenger 12's display 16 (area 20) or may only be able to see it partially (area 22). The passenger 12, whose head is located in area 24, has full visibility of the display 16. After the interaction with the infotainment system, the filter 18 can be deactivated again. Reference symbol list 10 drivers 12 Passenger / Passenger 14 Display and input system 16 Display and input device 18 switchable filters 20, 22, 24 visibility zones S1 to S9 process steps S1' alternative content of the driver's display and control unit V Video Y, N decisions in the procedure for queries QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] GB 2 583 890 A1

[0004] JP 2007 203 987 A2

[0005]

Images