SYSTEM AND METHOD FOR DETECTING A TARGET USER DEVICE FOR IMPROVED SAFETY IN AUTONOMOUS VEHICLES

Abstract

This disclosure relates to a system (102) and a method for detecting target user devices to ensure improved safety in autonomous vehicles (100). The system (102) comprises an image acquisition unit (110) for capturing images of one or more user devices (112) assigned to the user. The system (102) includes a control unit (104) to enable gaze detection in order to locate one or more user devices (112) based on the one or more images and to activate an autopilot mode in an autonomous vehicle (100) by the user. The control unit (104) generates at least one takeover action (TOA) cue when the autopilot mode is active. The control unit (104) detects the target user device (112-1) assigned to the user based on gaze detection.The control unit (104) dynamically transmits the at least one TOA notice to the target user device (112-1) assigned to the user using one or more adaptive channels to ensure improved safety in the autonomous vehicle (100).

Description

TECHNICAL AREA

[0001] This disclosure relates to the field of autonomous driving systems. In particular, this disclosure provides a system and a method for detecting a target user device in order to ensure improved safety in autonomous vehicles. BACKGROUND

[0002] Autonomous driving technology refers to the development and implementation of systems that enable vehicles to operate without human intervention. These autonomous systems utilize a combination of sensors, cameras, radar, light detection and ranging (LiDAR), and advanced algorithms to perceive the environment, make decisions, and control the vehicle.

[0003] In the field of autonomous driving, ensuring the driver's readiness to take over vehicle control is a crucial aspect. Furthermore, monitoring the driver's attention and engagement is essential, especially when the vehicle is operating in semi-autonomous mode (Level 3 and above). One of the primary challenges of autonomous driving is ensuring the driver remains attentive and ready to assume control when necessary. This scenario becomes particularly difficult when the driver is using multiple devices such as smartphones or other gadgets. Efficient detection of smartphone usage and timely notification of the activated device are vital to mitigate these risks and improve overall safety.

[0004] Many techniques have been developed to avoid the aforementioned problems. For example, patent US20210331690A1 describes systems and methods for notifying a user during autonomous driving. The system involves a vehicle with a communication device and an autonomous vehicle control unit (APU). The APU, which is communicatively coupled to the communication device, controls the vehicle in both autonomous and manual driving modes. In autonomous mode, the APU operates the vehicle, receives information about the vehicle's environment, and evaluates events based on this information. When an event is detected, the APU alerts the communication device, thus ensuring efficient transitions between driving modes and event response.

[0005] Another patent document, CN114454895A, describes a method, equipment, and system for the automatic takeover of a vehicle. The method comprises determining that a driver's mobile device is connected to a vehicle, sending a takeover request to the mobile device in an autonomous driving mode, wherein the takeover request serves to request a driver to take over driving the vehicle, receiving an operating instruction from a driver, and entering a driving mode in which the driver takes over driving the vehicle based on the operating instruction.

[0006] Patent CN114228728A describes a method, system, and software product for operating a vehicle with an automated driving function. The system relates to autopilot systems for vehicles with automated driving functions. It establishes communication between a mobile device and the vehicle's control unit. Furthermore, the system activates automated driving when it detects the connection. Additionally, under certain conditions, the system reminds the driver of the automated driving function via the mobile device. A smartphone enables L3 class automated driving functionality and improves safety and reliability.

[0007] Known systems and procedures for automated driving cannot effectively detect the use of multiple user devices or accurately identify the specific device the driver is using, resulting in delayed or missing takeover action (TOA) cues, which can lead to unsafe conditions.

[0008] Therefore, there is a need for a system and a method for detecting target user devices in order to ensure improved safety in autonomous vehicles by at least overcoming the problems mentioned above. SUBJECT OF THE PRESENT DISCLOSURE

[0009] A general objective of the present disclosure is to provide a system and a method for detecting a target user device in order to ensure improved safety in an autonomous vehicle.

[0010] One purpose of this disclosure is to ensure improved driver safety by guaranteeing prompt and accurate communication between the vehicle and the driver's personal devices, thereby reducing the risk of accidents and improving the overall effectiveness of autonomous driving technology.

[0011] Another purpose of this disclosure is to notify the driver in a timely manner so that he can take over control if necessary, in order to avoid potential accidents and ensure a smooth transition from autonomous to manual driving.

[0012] Another objective of the present disclosure is the accurate detection and localization of the user device stream with which the driver interacts, even when multiple devices are present, using advanced gaze detection mechanisms and dynamic quadrant mapping algorithms. SUMMARY

[0013] This disclosure relates to the field of autonomous driving systems. In particular, this disclosure provides a system and a method for detecting a target user device in order to ensure improved safety in an autonomous vehicle.

[0014] According to one aspect of the present disclosure, a system for detecting a target user device is described to ensure improved safety in autonomous vehicles. The system includes at least one image acquisition unit, which can be configured to capture one or more images of one or more user devices associated with at least one user in the autonomous vehicle. The system includes a control unit configured to enable gaze detection to locate one or more user devices based on the activation of an autopilot mode in the autonomous vehicle by the at least one user. The control unit is configured to generate at least one takeover action (TOA) cue in response to the activation of the autopilot mode and the gaze detection.The control unit is configured to detect the target user device assigned to the at least one user based on gaze detection. Furthermore, the control unit is configured to dynamically transmit the at least one TOA (Target User Approval) notification to the target user device assigned to the at least one user by using one or more adaptive channels to ensure enhanced safety in the autonomous vehicle. The at least one TOA notification is transmitted based on gaze detection and the connection status of the target user device assigned to the autonomous vehicle.

[0015] In some embodiments, the connection state can include a paired state and an unpaired state.

[0016] In some embodiments, the one or more adaptive channels may include a user interface (UI) of the target user device, a UI of the one or more user devices, and a main unit of the autonomous vehicle.

[0017] In some embodiments, the control unit can be configured for dynamic transmission of the at least one TOA message using one or more adaptive channels such that it transmits the at least one TOA message to the target user device when the target user device is operating in a paired state with the autonomous vehicle. The control unit can be configured to send the at least one TOA message to the one or more user devices available in the autonomous vehicle when the target user device is operating in an unpaired state with the autonomous vehicle. The control unit can be configured to display the at least one TOA message on a main unit of the autonomous vehicle based on the unavailability of the connection status of the target user device and the one or more user devices.

[0018] In some embodiments, the control unit can be configured to determine one or more gaze coordinates based on one or more parameters and gaze detection. The one or more parameters may include facial landmarks, eye localization, pupil coordination, and head posture estimation.

[0019] In some embodiments, the control unit can be configured to determine the spatial localization of one or more user devices in relation to the presence of at least one user and dynamically update the assignment of the target user device based on changes in the position of one or more user devices and / or the connection status.

[0020] In some embodiments, spatial localization can be calculated based on one or more factors, including a field of view (FoV) of the at least one image acquisition unit, a bounding frame filter associated with spatial localization, and one or more viewing coordinates of the one or more user devices.

[0021] In some embodiments, the control unit can be configured to compare the one or more viewing coordinates with the one or more spatial localizations of the one or more user devices in order to detect the target user device.

[0022] In some embodiments, the control unit can be configured to dynamically update a list of target user devices based on the availability, connection status, and operating state of one or more user devices.

[0023] In another aspect, a method is disclosed that is implemented by at least one image acquisition unit and a control unit to detect a target user device in order to provide enhanced safety in autonomous vehicles, including capturing one or more images from one or more user devices associated with at least one user in the autonomous vehicle. The method includes activating gaze detection to locate the one or more user devices based on the one or more images and the activation of an autopilot mode in the autonomous vehicle by at least one user. The method includes generating at least one TOA (Target Approach) indicator in response to the activation of the autopilot mode and the gaze detection. The method includes detecting the target user device associated with the at least one user based on the gaze detection.The method involves the dynamic transmission of at least one TOA (Target Approach to Autonomous Vehicle) indicator to the target user device assigned to the at least one user, using one or more adaptive channels to ensure enhanced safety in the autonomous vehicle. The at least one TOA indicator is transmitted based on gaze detection and the connection status of the target user device assigned to the autonomous vehicle.

[0024] Various objects, features, aspects and advantages of the subject matter according to the invention will become clearer from the following detailed description of preferred embodiments together with the accompanying drawing figures, in which the same numbers represent the same components. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings serve to further understand the present disclosure and are an integral part of this description. The drawings illustrate exemplary embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure. Fig. Figure 1 shows an exemplary block diagram of a system for detecting a target user device to provide improved safety in autonomous vehicles, according to the embodiments of the present disclosure. Fig. Figure 2 shows an exemplary block diagram of a control unit according to the embodiments of the present disclosure. Fig. Figures 3A-3B show exemplary block diagrams representing a training pipeline for implementing the detection of a target user device according to the embodiments of the present disclosure. Fig. Figure 4 shows a flowchart of an example method for detecting a target user device to provide improved safety in autonomous vehicles, according to the embodiments of the present disclosure. Fig. Figure 5 shows an example of a computer system in which or with which embodiments of the system according to the embodiments of the present disclosure can be implemented. DETAILED DESCRIPTION

[0026] A detailed description of the embodiments of the disclosure illustrated in the accompanying drawings follows. The embodiments are described in sufficient detail to clearly convey the disclosure. However, the necessary level of detail is not intended to limit foreseeable variations of embodiments; on the contrary, it is intended to cover all modifications, equivalents, and alternatives that fall within the scope of this disclosure as defined by the accompanying claims.

[0027] The embodiments described herein relate to the field of autonomous driving systems. In particular, the present disclosure provides a system and a method for detecting a target user device in order to ensure improved safety in autonomous vehicles.

[0028] Various embodiments of the present disclosure are described with reference to the Fig. 1-5 explained in more detail. 1-5.

[0029] Fig. Figure 1 shows an exemplary block diagram 100 of a system 102 for detecting target user devices to provide improved safety in autonomous vehicles, according to the embodiments of the present disclosure.

[0030] In Fig. Figure 1 shows a system 102 for detecting a target user device 112-1, which provides improved safety in autonomous vehicles 100. The system 102 can be used in advanced driver assistance systems (ADAS) and in the autonomous driving of vehicles 100. The system 102 can include, among other things, a control unit 104, at least one sensor 106, a data provision unit 108, and at least one image acquisition unit 110. The sensor 106 can be a remotely controlled device that continuously monitors the environment. The sensors 106 can detect the connectivity, movement, and location of the device. The sensor information can be used to trigger relevant notifications or actions, thus ensuring seamless interaction between the user and the vehicle's autonomous system. The control unit 104 can be communicatively coupled with the at least one sensor 106.The control unit 104 can be an electronic unit configured to process the data collected by the sensor 106 in order to generate real-time decisions based on the output of the sensor 106.

[0031] Furthermore, the data delivery unit 108 can be configured to collect, process, and deliver data to other systems or applications. The data delivery unit 108 can act as a bridge between data sources (e.g., sensors, databases, or external APIs) and data consumers by ensuring seamless and reliable data delivery. The data delivery unit 108 can include data validation, transformation, and formatting functions to meet the specific requirements of the system 102. The at least one image acquisition unit 110 can be configured to capture one or more images from one or more user devices 112, each assigned to at least one user.Furthermore, based on one or more images of the environment and the user's actions, the system 102 can identify the target user device 112-1 among one or more user devices 112-1, 112-2, 112-3 (here collectively referred to as "the user device 112"), which may include, among other things, a smartphone, a tablet, a portable device and the like.

[0032] The control unit 104 can be configured to activate gaze detection to locate one or more user devices 112 based on one or more images and the activation of an autopilot mode in an autonomous vehicle 100 by at least one user. The control unit 104 can be configured to generate at least one takeover action (TOA) cue in response to the activation of the autopilot mode and gaze detection. The control unit 104 can be configured to detect the target user device 112-1 assigned to the at least one user based on gaze detection and the one or more images.

[0033] The control unit 104 can be configured to dynamically transmit the at least one TOA (Target Operator Approval) notification to the target user device 112-1 assigned to the at least one user by using one or more adaptive channels to ensure enhanced safety in the autonomous vehicle. The at least one TOA notification is transmitted based on gaze detection and the connection status of the target user device 112-1 assigned to the autonomous vehicle 100. The one or more adaptive channels can include a user interface (UI) of the target user device 112-1, a UI of the one or more user devices 112, and a main unit 342 of the autonomous vehicle 100.

[0034] In some embodiments, the control unit 104 can be configured to transmit the at least one TOA notice to the target user device 112-1 when the target user device 112-1 is operated in the coupled state with the autonomous vehicle 100.

[0035] In some embodiments, the control unit 104 can be configured to transmit the at least one TOA message to the one or more user devices 112 available in the autonomous vehicle 100 when the target user device 112-1 is operating in an unpaired state with the autonomous vehicle 100. Furthermore, the control unit 104 can be configured to display the at least one TOA message on the main unit 346 of the autonomous vehicle 100 based on the unavailability of the connection status of the target user device 112-1 and the one or more user devices 112. Finally, the control unit 104 can be configured to compare the one or more gaze coordinates with the one or more spatial localizations of the one or more user devices 112 to detect the target user device 112-1.

[0036] In some embodiments, the control unit 104 can be configured to determine one or more gaze coordinates based on one or more parameters and gaze detection. The one or more parameters may include, among others, facial landmarks, eye localization, pupil coordination, head posture estimation, and the like.

[0037] In some embodiments, the control unit 104 can be configured to determine one or more spatial locations of the one or more user devices 112 relative to the presence of the at least one user. Furthermore, the control unit 106 can dynamically update the assignment of the target user device 112-1 based on changes in the position of the one or more user devices 112 and / or the connection status.

[0038] According to Fig. 2. The system 102 and / or the control unit 104 may contain the components shown in block diagram 200. For example, the control unit 104 may contain one or more processor(s) 202. The one or more processor(s) 202 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuits, and / or any devices that manipulate data based on operating instructions. Among other capabilities, the one or more processors 202 may be configured to retrieve and execute computer-readable instructions stored in a memory 204 of the control unit 104. The memory 204 may store one or more computer-readable instructions or routines that can be retrieved and executed to create or share data units via a network service.The memory 204 can include any non-volatile storage device, e.g., volatile memory such as Random Access Memory (RAM) or non-volatile memory such as Erasable Programmable Read-Only Memory (EPROM), Flash Memory, and the like.

[0039] In one embodiment, the system 102 may also include one or more interfaces 206. The interface(s) 206 may include a variety of interfaces, such as interfaces for data input and output devices, referred to as input / output (I / O) devices, storage devices, and the like. The interface(s) 206 may enable communication between the sensor 106, the image acquisition unit 110, and the control unit 104. The interface(s) 206 may also provide a communication path for one or more components of the system 102. Examples of such components include the processing machine(s) 208 and the database 210.

[0040] In one embodiment, the processing machine(s) 208 can be implemented as a combination of hardware and programming (e.g., programmable instructions) to implement one or more functions of the processing machine(s) 208. In the examples described here, such combinations of hardware and programming can be implemented in various ways. For example, the programming for the processing machine(s) 208 can consist of processor-executable instructions stored on a non-volatile, machine-readable storage medium, and the hardware for the processing machine(s) 208 can include a processing resource (e.g., a control unit) to execute such instructions. In other embodiments, the processing machine(s) 208 can be implemented by electronic circuits.Database 210 can contain data that is either stored or generated as a result of functionalities implemented by one of the components of processing machine(s) 208.

[0041] In some embodiments, the processing machine(s) 208 may comprise a gaze detection machine 212, a notification generation machine 214, a computation machine 216, and one or more other machines 218. The other machine(s) 218 ​​may implement functions that complement the applications / functions performed by the system 102.

[0042] The gaze detection machine 212 can be configured to identify and track the user's eye movements in order to determine their focus and intention. The gaze detection machine 212 uses input from the image acquisition unit 110 and the sensors 106 to calculate gaze coordinates in real time. The gaze detection machine 212 can be configured to map the user's attention to the target user device 112-1 or to one or more user devices 112 in the environment.

[0043] The notification generation machine 214 can be configured to generate and transmit alerts or messages to users by processing contextual data such as user intent, device status, or environmental conditions to determine the type and urgency of the notifications. The notifications are then formatted for display on the appropriate devices, such as smartphones, main units, or wearables.

[0044] The Computing Engine 216 can be configured to perform complex data processing and decision-making tasks to support system operation. The Computing Engine 216 can integrate inputs from multiple sensors, including eye-tracking and localization systems, to compute actionable outputs. The Computing Engine 216 enables real-time analysis and mapping, such as identifying the nearest device or prioritizing user commands.

[0045] Fig. Figures 3A-3B show an exemplary block diagram representing a training pipeline for implementing the detection of the target user device 112-1 according to the embodiments of the present disclosure.

[0046] In some embodiments, the system 102 can be configured to capture one or more images via the image acquisition unit 110. When blocking 302, a spatial localization mechanism can be configured to identify the position of one or more user devices 112 within an environment by using data from GPS, Bluetooth, Wi-Fi signals, and / or one or more sensors 106. The spatial localization mechanism can be configured to calculate one or more spatial coordinates relative to the user or a reference point in order to accurately track the location of the user device. As in Fig. As shown in Figure 3B, in block 302-1, the image acquisition unit 110 or the sensors 106 are used to capture the field of view (FoV) in order to determine the visual data of the environment. In block 302-2, the object detection model analyzes the field of view to identify one or more user devices 112 within the vehicle 100. In block 302-3, a bounding frame filtering model can be used to refine detection and isolate relevant objects by removing noise or unrelated objects. In block 302-4, the coordinates of the user device relative to the user or the environment are calculated to determine the location of the target user device 112-1.

[0047] In some embodiments, which relate to Fig. Referring to 3A, the control unit 104 in block 304 can be configured to perform gaze detection by analyzing the user's eye movements to determine their focus and / or attention within a specific area received by block 302. Furthermore, based on one or more images and sensor data, the control unit 104 can calculate one or more gaze coordinates in real time to map the user's intent. Additionally, gaze detection aids in identifying the device 1 12-1 associated with the user. As described in Fig. As shown in Figure 3B, in Block 304-1, one or more images are used to identify landmarks for facial recognition, capturing key facial features such as the eyes, nose, and mouth to provide a reference for further analysis. In Block 304-2, eye localization determines the precise position of the eyes within the detected facial region, enabling detailed tracking. In Block 304-3, a pupil coordination unit identifies the position of the pupils and maps pupil movement for a precise estimate of gaze direction. In Block 304-4, the head position is estimated based on the user's head orientation. In Block 304-5, the control unit 104 blocks gaze detection based on the calculated gaze coordinates of the user to determine where the user is looking, allowing the system to respond contextually.

[0048] In some embodiments, the detected locations of gazes and devices are aligned with the physical environment by the coordinate mapping unit in block 306. Inputs from sensors, localization algorithms, and geometric models received by block 304 are also integrated. The coordinate mapping unit ensures the accurate spatial orientation of one or more detected user devices 112.

[0049] In block 306-1, an estimation of the mean gaze vector can be performed based on gaze detection (starting in block 304-5). The mean gaze vector estimation can refer to the calculation of the user's average gaze direction based on gaze detection, such as eye movement or head position.

[0050] Block 306-2 allows the creation of an image quadrant based on the division of the visual interface into specific quadrants, providing a structured framework for gaze detection. Each quadrant corresponds to a segment of the display or target area and enables the assignment of the gaze vector determined in Block 306-1 to a specific region.

[0051] In block 306-3, a gaze quadrant estimation (g_quad) can assign the calculated gaze vector from block 306-1 to one of the predefined image quadrants. Furthermore, the result of the gaze quadrant estimation ensures precise localization of the user's gaze within the visual field. In block 306-4, a mobile phone quadrant estimation (c_quad) can determine the position of the user devices 112 within the same quadrant frame. Additionally, the mobile phone quadrant estimation 306-4 can identify the quadrant in which the device 112-1 might be located relative to the user's gaze.

[0052] In block 306-5, the control unit 104 can validate whether the viewing quadrant matches the quadrant of the mobile phone, and based on the alignment, the user can look at the target user devices 112-1.

[0053] In some embodiments, the control unit 104 in block 308 can identify the target user device 112-1, which is associated with the user, based on block 306-5. Furthermore, the target user device 112-1 is detected by combining gaze detection, spatial localization, and contextual input. The recognition process adapts dynamically based on the user's intent and the availability of the target user device 112-1.

[0054] In some embodiments, a communication unit in Blockieren 310 enables the exchange of information between the system components and the user devices 112. The communication unit relies on various protocols to enable seamless data transmission and connectivity of the device.

[0055] In some implementations, an Internet Protocol (blocking IPv4 and IPv6) in block 312 manages IP addressing for network communication within System 102. IPv4 provides support for older systems, while IPv6 offers scalability and advanced features.

[0056] In some embodiments, a secure link is established in block 314 with the target user device 112-1, which enables the transmission of notifications, commands or data as needed based on the network communication from block 312.

[0057] In some embodiments, blocks HMI 316-1 and HMI 316-2 (Human-Machine Interface) enable user interaction via visual displays, touchscreens, or voice commands based on the secure connection of block 314. Block 316-1 provides real-time information and feedback on user input or system status to the spatial localization mechanism (block 302). Block 316-2 initiates interaction with the target user device 112-1 by activating user-friendly controls and customizable interfaces.

[0058] In some embodiments, the Data Processing Input (DPI) unit in block 318 blocks the preprocessing of the input data by using the HMI blocks 316-1, 316-2 to perform the analysis and processing.

[0059] In some embodiments, a sensor unit in Block 320 integrates the data from one or more sensors 106 to create a coherent picture of the environment. The one or more sensors 106 may include, among others, a stereo multi-purpose camera (SMPC) 106-1, a radio detection and ranging (RADAR) 106-2, a light detection and ranging (LiDAR) 106-3, a high-definition (HD) map 106-4, and the like. The sensor fusion unit eliminates redundancies and improves detection accuracy.

[0060] In some embodiments, a horizon manager can be configured in block 322 to manage the vehicle's perception of the environment and anticipate changes in the environment based on the sensor fusion unit from block 320. The horizon manager analyzes sensor data to manage road features, obstacles, and trajectories.

[0061] In some embodiments, a vehicle localization unit in block 324 can be configured to determine the vehicle's precise position in its environment, based on the horizon management from block 322. The vehicle localization unit uses data from GPS and environmental maps to calculate the location.

[0062] In some embodiments, a drive control unit in block 326 can be configured to activate autonomous driving mode in vehicle 100. The control unit transfers control from the user to the vehicle's intelligent system. This blocking ensures a safe and smooth switching process by verifying that all systems are operational before activation.

[0063] In some embodiments, Blockieren 328 can be configured to include a Human-Machine Output (HMO) to manage the transfer of control between the vehicle and the user. The HMO ensures a smooth and safe handover process in dynamic scenarios. The HMO provides the user with necessary warnings or instructions and verifies that both the user and the system are ready for the transition. An effective HMO ensures uninterrupted control and safety.

[0064] In some embodiments, the Autopilot mode in Block 330 manages the disabling of hands-free driving assistance, based on Block 328. The Autopilot mode issues incremental warnings to inform the user of the control unit's reactivation and ensures the user has sufficient time to safely resume control. The Autopilot mode may adjust system functions during the transition to maintain stability.

[0065] In some embodiments, a Takeover Action (TOA) unit activated in block 332 triggers warnings or actions to alert the user to take control of the vehicle 100 by using the HAF in block 330. The TOA unit ensures timely and clear communication with the user and plays a crucial role in maintaining safety during autonomous operation by minimizing risks in emergency scenarios.

[0066] In some embodiments, an input / output (I / O) unit in block 334 enables the exchange of data and commands between the system components and one or more user devices 112, based on the TOA in block 332. The I / O unit ensures efficient communication for system inputs, e.g., user commands, and outputs, such as notifications.

[0067] In some embodiments, a DPO (Data Processing Output) unit in block 336 blocks the vehicle's settings and interactions based on user preferences. User preferences may include driving habits, comfort settings, interaction history, and similar information.

[0068] In some embodiments, a sensor warning module in block 338 warns the system 102 and the user of potential hazards or anomalies detected by the sensors 106. The sensor warning module processes data from multiple sensors 106 to detect risks such as obstacles, environmental changes, system malfunctions, and the like.

[0069] In some embodiments, in Blockieren 340, a Rest of Vehicle (ROV) may refer to the interaction between one or more components of the vehicle to ensure smooth operation.

[0070] In some embodiments (see Fig. 3B) Block 342 transmits a Targeted Takeover Action (TOA) alert to the target device 122-1, which may be a targeted warning or a command based on gaze detection, device localization, or contextual input. Blocking block 342 ensures that the notification reaches the user promptly and efficiently.

[0071] In some embodiments, the notification sending block in block 344-1 is responsible for generating and transmitting alerts or messages to the desired device or user. It formats the notification based on system requirements and selects the optimal delivery channel, such as mobile devices, main units, or wearables. The notification receiver's block 344-2 is coupled with block 342, which can process incoming alerts or messages transmitted by the notification sender in block 344-1. Block 344-2 verifies the integrity and relevance of the received notification before presenting it to the user.

[0072] In some embodiments, a message on the main unit in block 346 blocks critical system messages or notifications directly on the vehicle's center console from TOA notice 342.

[0073] Referring to Fig. 4. A method 400 for detecting a target user device 122-1, in order to ensure improved safety in autonomous vehicles 100, can comprise a variety of blocks 402-408. The method 400 can be implemented by the system 102 or its control unit 104.

[0074] In block 402, the method 400 comprises the capture of one or more images from one or more user devices 112, which are assigned to at least one user in the autonomous vehicle 100, by at least one image capture unit 110.

[0075] In block 404, the procedure 400 includes the activation of gaze detection to locate the one or more user devices 112 based on the one or more images and the activation of an autopilot mode in the autonomous vehicle 100 by the at least one user.

[0076] In Blocking 406, the procedure 400 includes detecting the target user device 112-1, which is assigned to the at least one user, based on gaze detection.

[0077] In Block 408, Method 400 comprises the dynamic transmission of the at least one TOA indicator to the target user device 112-1 assigned to the at least one user, using one or more adaptive channels to ensure enhanced safety in the autonomous vehicle 100. The at least one TOA indicator is transmitted based on gaze detection and the connection status of the at least one target user device 112-1 assigned to the autonomous vehicle 100.

[0078] System 102 and Procedure 400 can be implemented in a computer system. Fig.Figure 5 shows a block diagram of a computer system 500 comprising an external storage device 510, a bus 520, main memory 530, read-only memory 540, mass storage device 550, a communication port 560, and a processor 570. A person skilled in the art will understand that the system 500 may comprise more than one processor 570 and communication ports 560. The processor 570 may include various modules associated with the embodiments of this disclosure. The communication port 560 may be a recommended standard 232 port for use with a modem-based dial-up connection, a 10 / 100 Ethernet port, a Gigabit or 10 Gigabit port over copper or fiber optic cable, a serial port, a parallel port, or other existing or future ports. The port 560 may be selected depending on a network, such as...a Local Area Network (LAN), a Wide Area Network (WAN) or any other network to which the System 500 is connected.

[0079] In one embodiment, the memory 530 can be a RAM or other dynamic storage device generally known in the art. The read-only memory (ROM) 540 can be any static device, e.g., a programmable read-only memory (PROM) for storing static information. The mass storage 550 can be any current or future mass storage solution that can be used to store information and / or instructions. Exemplary mass storage solutions include, but are not limited to, parallel Advanced Technology Attachment (PATA) or serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., with Universal Serial Bus (USB) and / or FireWire interfaces), one or more optical disks, RAID (Redundant Array of Independent Disks) storage, e.g., an array of hard disks (e.g., SATA arrays).

[0080] In one embodiment, the bus 520 provides communication between the processor(s) 570 and the other storage, retention, and communication blocks. The bus 520 may be, for example, a Peripheral Component Interconnect (PCI) / PCI-Extended (PCI-X) bus, a Small Computer System Interface (SCSI), a USB bus, or similar, for connecting expansion cards, drives, and other subsystems, as well as other buses, such as a front-end bus (FSB) that connects the processor 570 to the computer system 500.

[0081] In another embodiment, operator and management interfaces, such as a display device, a keyboard, and a cursor control unit, can also be connected to the bus 520 to support direct operator interaction with the computer system 500. Other operator and management interfaces can be provided via network connections connected through the communication port 560. In some embodiments, the external storage device 510 can be any type of external hard disk drive, floppy disk drive, Compact Disc - Read Only Memory (CD-ROM), Compact Disc - Re-Writable (CD-RW), or Digital Video Disc - Read Only Memory (DVD-ROM). The components described above serve only to illustrate various possibilities. The computer system 500 mentioned is not intended to limit the scope of this disclosure in any way.

[0082] While the foregoing describes various embodiments of the present disclosure, other and further embodiments of the present disclosure may be developed without departing from the basic scope. The scope of the present disclosure is determined by the following claims. The present disclosure is not limited to the described embodiments, versions, or examples that are included to enable a person with ordinary technical knowledge to manufacture and use the present disclosure when combined with the information and knowledge available to such a person. BENEFITS OF THE PRESENT DISCLOSURE

[0083] This disclosure makes it possible to detect target user devices in order to ensure improved safety in autonomous vehicles.

[0084] This disclosure ensures prompt and accurate communication between the vehicle and the driver's personal devices, thereby reducing the risk of accidents and improving the overall effectiveness of autonomous driving technology.

[0085] Another objective of this disclosure is to notify the driver in a timely manner so that he can take over control if necessary, thereby avoiding potential accidents and ensuring a smooth transition from autonomous to manual driving.

[0086] Another objective of the present disclosure is to accurately detect and locate the stream of user device with which the driver interacts, even when multiple devices are present, using advanced gaze detection and dynamic quadrant mapping algorithms. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 20210331690A1

[0004] CN 114454895A

[0005] CN 114228728A

[0006]

Claims

[1] System (102) for detecting a target user device (112-1) to provide enhanced safety in an autonomous vehicle (100), the system (102) comprising: at least one image acquisition unit (110) configured to capture one or more images from one or more user devices (112) assigned to at least one user in the autonomous vehicle (100); and a control unit (104) configured to: eye-tracking enables the localization of one or more user devices (112) based on one or more images and activation of an autopilot mode in the autonomous vehicle (100) by the at least one user; generates at least one Takeover Action (TOA) cue that responds to the activation of autopilot mode and gaze detection; the target user device (112-1) assigned to at least one user is detected based on gaze detection; and dynamically transmits the at least one TOA notice to the target user device (112-1) assigned to the at least one user using one or more adaptive channels to provide enhanced safety in the autonomous vehicle (100), wherein the at least one TOA notice is transmitted based on gaze detection and a connection status of the target user device (112-1) assigned to the autonomous vehicle (100). [2] System (102) according to claim 1, wherein the connection state comprises: a paired state and an unpaired state. [3] System (102) according to claim 1, wherein the one or more adaptive channels comprise: a user interface (UI) of the target user device (112-1), a user interface (UI) of the one or more user devices (112) and a main unit (342) of the autonomous vehicle (100). [4] System (102) according to claim 1, wherein the control unit (104) is configured to dynamically transmit the at least one TOA indicator using one or more adaptive channels: to transmit at least one TOA notification to the target user device (112-1) when the target user device (112-1) is operated in a coupled state with the autonomous vehicle (100); to transmit at least one TOA notice to the one or more user devices (112) available in the autonomous vehicle (100) when the target user device (112-1) is operating in an unpaired state with the autonomous vehicle (100); and to display at least one TOA notification on a main unit (342) of the autonomous vehicle (100) based on the unavailability of the connection status of the target user device (112-1) and the one or more user devices (112). [5] System (102) according to claim 1, wherein the control unit (104) is configured to determine one or more gaze coordinates based on one or more parameters and gaze detection, and wherein the one or more parameters include: facial landmarks, eye localization, pupil coordination and head posture estimation. [6] System (102) according to claim 1, wherein the control system (104) is configured to determine a spatial localization of the one or more user devices (112) with respect to the presence of the at least one user and dynamically updates the mapping of the target user device (112-1) based on changes in at least one position of the one or more user devices (112) and the connection status. [7] System (102) according to claim 6, wherein the spatial localization is calculated on the basis of one or more factors comprising: a field of view (FoV) of the at least one image acquisition unit (110), a bounding frame filter associated with the spatial localization, and one or more viewing coordinates of the one or more user devices (112). [8] System (102) according to claim 7, wherein the control unit (104) is configured to compare the one or more viewing coordinates with the spatial localization of the one or more user devices (112) in order to detect the target user device (112-1). [9] System (102) according to claim 1, wherein the control system (104) is configured to dynamically update a target user device list based on the availability, connection status and operating state of one or more user devices (112). [10] Method (400) for detecting a target user device (112-1) to provide enhanced safety in an autonomous vehicle (100), the method (400) comprising: Capturing one or more images from one or more user devices (112) assigned to at least one user in the autonomous vehicle (100) by at least one image capture unit (110); Enabling, by means of a control unit (104), eye-tracking detection for locating one or more user devices (112) based on one or more images and activation of an autopilot mode in the autonomous vehicle (100) by the at least one user; Generating at least one Takeover Action (TOA) indication by the control unit (104) in response to the activation of autopilot mode and gaze detection; Detection of the target user device (112-1) assigned to the at least one user by the control unit (104) based on eye-tracking detection; and dynamic transmission of the at least one TOA notice to the target user device (112-1) assigned to the at least one user by the control unit (104) using one or more adaptive channels to provide enhanced safety in the autonomous vehicle (100), wherein at least one TOA indication is transmitted based on gaze detection and a connection status of the target user device (112-1) assigned to the autonomous vehicle (100).

Images