Method for operating a vehicle, computer-readable medium, system, and vehicle

Abstract

The invention relates to a method for operating a vehicle, the method comprising: detecting the use of a mobile device on the driver's wrist by the vehicle, wherein the detection of the use of the mobile device on the driver's wrist by the vehicle comprises: detecting the use of the detected mobile device on the driver's wrist depending on an image of an activated display of the detected mobile device on the driver's wrist, wherein the image of the activated display of the detected mobile device on the driver's wrist is captured with an RGB camera of the interior camera system; and adapting a vehicle function depending on the detected use of the mobile device on the driver's wrist by the vehicle.

Description

The invention relates to a method for operating a vehicle. The invention further relates to a computer-readable medium for operating a vehicle, a system for operating a vehicle, and a vehicle comprising the system for operating the vehicle. Various methods for detecting driver distraction caused by smartphone use in a vehicle are known in the prior art. It is also known to detect smartphone use inside a vehicle. For example, smartphone use can be detected using a 3D body pose model. However, methods for detecting smartwatch use inside a vehicle are not known in the prior art. DE 10 2022 129 845 A1 describes a method for controlling an output unit to output driving-related information to a vehicle occupant. DE 10 2017 215 404 A1 describes a method for controlling a mobile user device of an occupant of a vehicle. DE 10 2015 110 348 A1 describes a human-machine interface for transmitting warnings and messages to a driver. It is therefore an object of the invention to operate a vehicle more efficiently. In particular, an object of the invention is to detect driver distraction more efficiently. This problem is solved by the features of the independent claims. Advantageous embodiments and further developments of the invention result from the dependent claims. According to a first aspect, the invention is characterized by a method according to claim 1 for operating a vehicle. The method can be a computer-implemented method and / or a control unit-implemented method. The vehicle can be a motor vehicle. The method comprises detecting the use of a mobile device on the driver's wrist by the vehicle, wherein the detection of the use of the mobile device on the driver's wrist by the vehicle includes recognizing the use of the detected mobile device on the driver's wrist based on an image of an activated display of the detected mobile device on the driver's wrist. Furthermore, the method comprises adapting a vehicle function based on the detected use of the mobile device on the driver's wrist by the vehicle. Advantageously, the procedure can be operated more securely if the use of a mobile device on a wrist of the vehicle is detected. According to an advantageous embodiment, the mobile device on the driver's wrist can be a smartwatch. Alternatively, the mobile device can be integrated into a piece of clothing and / or jewelry, which the driver wears on their wrist. This allows the vehicle to be operated more efficiently when a smartwatch is used. According to a further advantageous embodiment, the vehicle's detection of the use of a mobile device on the driver's wrist can include capturing the driver's gaze direction and / or duration towards the mobile device on the driver's wrist using an interior camera system, and the vehicle's detection of the use of the mobile device on the driver's wrist depending on the driver's gaze direction and / or duration. This allows for efficient detection of the use of a mobile device on the driver's wrist. According to a further advantageous embodiment, the vehicle's detection of the driver's use of a mobile device on their wrist can include capturing the driver's interior camera system and / or microphone with a voice and / or touch gesture-based interaction, and detecting the use of the mobile device on the driver's wrist based on the driver's voice and / or gesture-based interaction. This allows for efficient detection of the driver's use of a mobile device on their wrist. According to a further advantageous embodiment, the vehicle's detection of the driver's use of a mobile device on their wrist can include displaying a message on a screen inside the vehicle, capturing the driver's gaze on the mobile device on their wrist by the vehicle's interior camera system while the message is being displayed, and detecting the use of the mobile device on the driver's wrist based on the driver's gaze on the mobile device on their wrist while the message is being displayed. This allows for efficient detection of the driver's use of a mobile device on their wrist. According to a further advantageous embodiment, the vehicle's detection of the driver's use of a mobile device on their wrist can comprise the vehicle's detection of a communication activity for sending a message to the mobile device on the driver's wrist via a wireless communication interface between the vehicle and the mobile device on the driver's wrist, and the vehicle's detection of the use of the mobile device on the driver's wrist in relation to this communication activity for sending a message to the mobile device on the driver's wrist via the wireless communication interface between the vehicle and the mobile device on the driver's wrist. This allows for efficient detection of the driver's use of a mobile device on their wrist. According to a further advantageous embodiment, the vehicle's detection of mobile device use on the driver's wrist can comprise the detection of a vibration and / or vibration pattern of the mobile device on the driver's wrist by a microphone in the vehicle's interior, and the detection of mobile device use on the driver's wrist based on the detected vibration and / or vibration pattern. This allows for efficient detection of mobile device use on the driver's wrist. According to a further advantageous embodiment, the vehicle's detection of the use of the mobile device on the driver's wrist can include capturing the mobile device on the driver's wrist with an infrared camera of the interior camera system, wherein the infrared camera captures the mobile device on the wrist using technical data of the mobile device, in particular computer-aided design data of the mobile device. Furthermore, the vehicle's detection of the use of the mobile device on the driver's wrist can include detecting the use of the captured mobile device on the driver's wrist based on an image of an activated display of the captured mobile device on the driver's wrist, wherein the image of the activated display of the captured mobile device on the driver's wrist is captured with an RGB camera of the interior camera system.This allows the use of a mobile device on the driver's wrist to be efficiently detected. According to a further advantageous embodiment, adapting the vehicle function based on the detected use of the mobile device on the driver's wrist can include adjusting a warning threshold and / or an intervention threshold of the vehicle function depending on the detected use of the mobile device, wherein, in particular, a warning threshold for the use of a watch is adjusted if the use of a smartwatch is detected. This allows the vehicle function to be adapted efficiently, thus enabling safer vehicle operation. According to a further advantageous embodiment, adapting the vehicle function based on the detected use of the mobile device on the driver's wrist can further include providing a message to the mobile device on the driver's wrist via a display within the driver's field of vision, wherein the display within the driver's field of vision is preferably determined by the vehicle's interior camera system. This allows the vehicle function to be adapted efficiently, thus enabling safer vehicle operation. According to a further aspect, the invention is characterized by a computer-readable medium according to claim 11 for operating a vehicle, wherein the computer-readable medium comprises instructions which, when executed on a computer and / or a control unit of the vehicle, execute the method described above. According to another aspect, the invention is characterized by a system according to claim 12 for operating a vehicle, wherein the system is configured to perform the method described above. According to another aspect, the invention is characterized by a vehicle according to claim 13 comprising the system described above for operating the vehicle. Further features of the invention will become apparent from the claims, the figures, and the description of the figures. All features and combinations of features mentioned above in the description, as well as those subsequently mentioned in the description of the figures and / or shown in the figures alone, are usable not only in the combinations specified, but also in other combinations or individually. A preferred embodiment of the invention is described below with reference to the accompanying drawings. Further details, preferred configurations, and further developments of the invention will be derived from these drawings. Specifically, Fig. 1 schematically shows an exemplary method for operating a vehicle, and Fig. 2 shows an exemplary system for operating a vehicle. Figure 1 shows in detail an exemplary method 100 for operating a vehicle. The method 100 can detect the use of a mobile device on the wrist of a driver of a vehicle 102. The mobile device on the driver's wrist can be a smartwatch. The use of the smartwatch can include reading information on a display of the smartwatch, performing a touch gesture on a touch-sensitive display of the smartwatch, pressing and / or touching a control element of the smartwatch, and / or voice-based interaction with a microphone of the smartwatch. The use of a mobile device on the driver's wrist, particularly a smartwatch, can be detected using one or more of the following methods. For example, the smartwatch can be detected using an image recognition method employed by an interior camera system. This involves comparing an image of the smartwatch with a smartwatch image database to identify the smartwatch on the driver's wrist. The smartwatch image database can be stored on the vehicle and / or on an external server. Additionally or alternatively, smartwatch use can be detected by measuring the driver's gaze duration. The vehicle's interior camera system can determine this duration, focusing on the smartwatch and / or the driver's wrist. If the detected gaze duration exceeds a predefined threshold, the vehicle can recognize smartwatch use. This predefined threshold could, for example, be a typical duration of gaze at a conventional watch. Additionally or alternatively, smartwatch usage can be detected based on one or more interactions with the smartwatch. An interaction with a smartwatch could be, for example, scrolling, swiping, or sliding on the smartwatch's touchscreen. Additionally or alternatively, smartwatch usage can be detected by the vehicle's interior camera system when an activation movement causes the smartwatch's display to brighten. Additionally or alternatively, smartwatch usage can be detected based on one or more voice interactions with the smartwatch's microphone. Additionally or alternatively, smartwatch usage can be detected based on the driver's reaction to a message displayed on the vehicle's screen. For example, the message could be displayed on a central screen for a predetermined period.In response to the displayed message, the system can detect if the driver glances at the smartwatch during the same predefined time period using the vehicle's interior camera system. If the interior camera system detects the driver glancing at the smartwatch during this same predefined time period, the system can recognize the smartwatch's use. Additionally or alternatively, the vehicle can determine smartwatch usage by analyzing Bluetooth communication messages between the smartwatch and the vehicle. If the message analysis shows a predefined minimum frequency of messages exchanged between the smartwatch and the vehicle, this can be considered representative of smartwatch usage. Additionally or alternatively, smartwatch usage can be detected by the vehicle's microphone via a vibration signal from the smartwatch upon receiving a message. Method 100 can adapt a vehicle function based on the vehicle's detected use of the mobile device on the driver's wrist. Using a smartwatch can lead to a higher cognitive load compared to wearing a conventional watch. This increased cognitive load can result in reduced attention and / or distraction for the driver. Therefore, Method 100 can adapt the vehicle function to account for the driver's reduced attention and / or distraction when using a smartwatch. Furthermore, the vehicle function can display a message from the smartwatch on a vehicle display, making the message easier for the driver to read with less distraction. In detail, Fig. 2 shows an exemplary system 200 for operating a vehicle 202. The system 200 can execute the method 100 described above in Fig. 1. The system 200 can include an interior camera system 204. The interior camera system 204 can detect a mobile device 206 on the wrist of a driver 208 of the vehicle. Furthermore, the interior camera system 204 can, for example, detect the use of the mobile device 206 based on an interaction between the driver 208 and the mobile device 206. If the interior camera system 204 detects the use of the mobile device 206, the interior camera system 204 can adjust a vehicle function 210. Advantageously, the method and the system can operate the vehicle more efficiently. Driver distraction can be effectively detected when a mobile device is worn on the driver's wrist, and the vehicle's functions can be adjusted accordingly. The vehicle can be operated more safely with reduced driver distraction. Reference symbol list 100 Procedure 102 Detecting the use of a mobile device on a driver's wrist 104 Adapting a vehicle function 200 System 202 Vehicle 204 Interior camera system 206 Smartwatch 208 Driver

Claims

Method (100) for operating a vehicle (202), the method (100) comprising: detecting (102) the use of a mobile device on the wrist of a driver (208) by the vehicle (202), wherein the detection of the use of the mobile device on the wrist of the driver (208) by the vehicle (202) comprises: detecting the use of the detected mobile device on the wrist of the driver (208) depending on an image of an activated display of the detected mobile device on the wrist of the driver (208), wherein the image of the activated display of the detected mobile device (206) on the wrist of the driver (208) is captured with an RGB camera of an interior camera system (204); and adapting (104) a vehicle function depending on the detected use of the mobile device on the wrist of the driver (208) by the vehicle (202). Method (100) according to claim 1, wherein the mobile device on the wrist of the driver of the vehicle is a smartwatch (206). Method (100) according to one of the preceding claims, wherein the detection (102) of the use of the mobile device on the driver's (208) wrist by the vehicle (202) comprises: detecting a direction of gaze and / or duration of gaze of the driver (208) towards the mobile device on the driver's (208) wrist by an interior camera system (204) of the vehicle (202); and detecting the use of the mobile device on the driver's (208) wrist depending on the direction of gaze and / or duration of gaze of the driver (208) towards the mobile device on the driver's (208) wrist by the vehicle (202). Method (100) according to any of the preceding claims, wherein the detection (102) of the use of the mobile device on the driver's (208) wrist by the vehicle (202) comprises: detecting a speech and / or gesture-based interaction of the driver (208) of the vehicle (202) with the mobile device on the driver's (208) wrist by the vehicle's (202) interior camera system (204) and / or a microphone of the vehicle (202); and detecting the use of the mobile device on the driver's (208) wrist depending on the speech and / or gesture-based interaction of the driver (208) of the vehicle (202) with the mobile device on the driver's (208) wrist by the vehicle (202). Method (100) according to any of the preceding claims, wherein the detection (102) of the use of the mobile device on the driver's (208) wrist by the vehicle (202) comprises: providing a message on a display in an interior of the vehicle (202); detecting the driver's (208) gaze at the mobile device on the driver's (208) wrist by the vehicle's (202) interior camera system (204) while the message is being provided on the display in the interior of the vehicle (202); and detecting the use of the mobile device on the driver's (208) wrist depending on the driver's (208) gaze at the mobile device on the vehicle's (202) wrist while the message is being provided on the display in the interior of the vehicle (202) by the vehicle (202). Method (100) according to any of the preceding claims, wherein the detection (102) of the use of the mobile device on the driver's wrist (208) by the vehicle (202) comprises: detecting a communication activity for providing a message to the mobile device on the driver's wrist (208) via a wireless communication interface between the vehicle (202) and the mobile device on the driver's wrist (208) of the vehicle (202) by the vehicle (202); and detecting the use of the mobile device on the driver's wrist (208) depending on the communication activity for providing a message to the mobile device on the driver's wrist (208) via the wireless communication interface between the vehicle (202) and the mobile device on the driver's wrist (208) of the vehicle (202) by the vehicle (202). Method (100) according to any of the preceding claims, wherein the detection of the use of the mobile device on the driver's wrist by the vehicle comprises: detecting a vibration and / or a vibration pattern of the mobile device on the driver's wrist (208) of the vehicle (202) by a microphone in the interior of the vehicle (202); and detecting the use of the mobile device on the driver's wrist (208) depending on the detected vibration and / or the detected vibration pattern of the mobile device on the driver's wrist (208) of the vehicle (202). Method (100) according to one of the preceding claims, wherein the detection (102) of the use of the mobile device on the driver's wrist (208) by the vehicle (202) comprises: detecting the mobile device on the driver's wrist (208) with an infrared camera of the interior camera system (204), wherein the infrared camera detects the mobile device on the wrist using technical data of the mobile device, in particular computer-aided design data of the mobile device. Method (100) according to one of the preceding claims, wherein the adaptation (104) of the vehicle function depending on the detected use of the mobile device on the wrist of the driver (208) of the vehicle (202) by the vehicle (202) comprises: adapting a warning threshold and / or an intervention threshold of the vehicle function depending on the detected use of the mobile device, wherein in particular a warning threshold for the use of a watch is adapted to the vehicle function if the use of a smartwatch (206) is detected. Method (100) according to one of the preceding claims, wherein the adaptation (104) of the vehicle function depending on the detected use of the mobile device on the driver's wrist by the vehicle further comprises: providing a message to the mobile device on the driver's wrist on a display in a field of view of the driver of the vehicle, wherein the display in the driver's field of view is preferably determined by the vehicle's interior camera system. Computer-readable medium for operating a vehicle (202), wherein the computer-readable medium comprises instructions which, when executed on a computer and / or a control unit of the vehicle (202), execute the method (100) according to any one of claims 1 to 10. System (200) for operating a vehicle (202), wherein the system (200) is configured to perform the method (100) according to any one of claims 1 to 10. Vehicle (202) comprising the system (200) for operating the vehicle (202) according to claim 12.

Images