Apparaus and method for providing a driver alert
A control system using existing vehicle components to detect and alert drivers to external objects outside their attention region, enhancing safety by minimizing additional hardware and optimizing alert relevance.
Patent Information
- Application Number
- GB2025005739
- Authority / Receiving Office
- GB · GB
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-04-16
- Publication Date
- 2026-11-11
AI Technical Summary
Existing systems fail to effectively alert drivers to external objects that fall outside their projected attention region, leading to potential hazards due to the driver's unawareness of these objects.
A control system that receives object and gaze data to determine if external objects are outside the driver's attention region, generating alerts through a driver alert system to draw attention to these objects, utilizing existing vehicle components like driver monitoring and assistance systems to minimize additional hardware requirements.
Accurately alerts drivers to external objects they may have missed, reducing the risk of collisions by ensuring timely awareness of potential hazards without distracting the driver with unnecessary alerts.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
TECHNICAL FIELD The present disclosure relates to an apparatus and method for providing a driver alert. Aspects of the invention relate to a control system, to a system, to a vehicle and to a method. BACKGROUND It is known to detect an object, such as another vehicle or pedestrian, in an environment of a vehicle and to provide an alert to a driver of the vehicle. It is an aim of the present invention to address one or more of the disadvantages associated with the prior art. SUMMARY OF THE INVENTION Aspects and embodiments of the invention provide a control system, a system, a vehicle, and a method as claimed in the appended claims. According to an aspect of the present invention there is provided a control system arranged to receive object data indicative of a location of one or more objects in an environment external to a vehicle, to receive gaze data indicative of a direction of gaze of a driver of the vehicle, and to determine, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of the driver. According to an aspect of the present invention there is provided a control system for controlling a driver alert system of a vehicle. The control system comprises one or more controllers collectively comprising at least one electronic processor having an electrical input for receiving an input signal; and at least one memory device electrically coupled to the at least one electronic processor and having instructions stored therein; and wherein the at least one electronic processor is configured to access the at least one memory device and execute the instructions thereon so as to: receive object data indicative of a location of one or more objects in an environment external to the vehicle; receive gaze data indicative of a direction of gaze of a driver of the vehicle; and determine, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of the driver. According to an aspect of the present invention there is provided a control system for controlling a driver alert system of a vehicle, the control system comprising one or more processors collectively configured to receive object data indicative of a location of one or more objects in an environment external to the vehicle, receive gaze data indicative of a direction of gaze of a driver of the vehicle, determine, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of the driver, and output a control signal to the driver alert system of the vehicle to generate an alert if any of the one or more objects fall outside of the projected driver attention region of the driver. Advantageously, the control system can determine whether there are any objects outside the vehicle (external objects) that the driver of the vehicle may not have noticed. These objects can be indicated to the driver. Advantageously, the driver of the vehicle is alerted to the presence of one or more external objects that they may not have already noticed i.e. they fall outside of the projected driver attention region. The driver is made aware of the one or more objects. Optionally, the gaze data is received from a driver monitoring system and / or cabin monitoring system associated with the vehicle. Advantageously, driver monitoring systems and cabin monitoring systems are generally incorporated in modern vehicles so the gaze data may be readily obtained. This reduces the number of additional components required for implementation and the associated expenses. Optionally, the driver monitoring and / or cabin monitoring system comprises one or more cameras arranged to output image data comprising an image of at least a portion of the driver. The driver monitoring and / or cabin monitoring system is arranged to determine the gaze data in dependence on the image data. The control system comprises an input arranged to receive the gaze data. Optionally, the object data is received from a driver assistance system of the vehicle. Advantageously, driver assistance systems are generally incorporated into vehicles so the object data can be readily obtained. That is, it is not necessary to incorporate or associate additional components into the control system to obtain the object data, which reduces the overall cost and complexity of implementation. Optionally, the object data is determined in dependence on an output of one or more sensors associated with the vehicle. Advantageously, the one or more sensors can provide an accurate indication of the objects in the environment of the vehicle, even if the objects are outside the driver’s field of view. Furthermore, sensor systems are frequently incorporated into vehicles so it is not necessary to incorporate additional components to obtain the object data. Optionally, the alert is at least one visual indicator corresponding to the location of the one or more objects. Advantageously, the driver’s attention may be rapidly drawn to the location of the one or more objects so that they are quickly made aware of the presence and position of these objects. The driver can then adjust their driving in a manner appropriate to the situation if necessary. Optionally, the location is such that the visual indicator is visually associated with the any of the one or more objects falling outside of the projected driver attention region of the driver. Advantageously, the visual indicator is generated for objects falling outside the projected driver attention region of the driver, so the driver is not unnecessarily alerted to objects of which they are likely already aware. The driver does not become distracted or fatigued by superfluous indicators so is more likely to pay attention when an visual indicator is output. Optionally, the gaze data is indicative of a location of a head of the driver of the vehicle, and the projected driver attention region is projected from the location of the head of the driver in the gaze direction. Advantageously, this provides an accurate indication of a projected driver attention region of the driver. Even within a same vehicle, different drivers will have different head positions due to variations of, for example, driver height, slouch, seating position and lean etc. A head of a single driver is also likely to change positions while driving as the driver shifts position, moves side to side or forward to back, tilts or moves their head, and looks around etc. Using a measured or determined location of a driver’s head to determine a projected driver attention region improves the accuracy of the projected driver attention region and therefore enables a more accurate determination of whether an object falls within or outside the gaze of the driver. Optionally, determining whether any of the one or more objects falls outside of the projected driver attention region of the driver comprises: determining one of a plurality of predetermined driver attention regions as a current driver attention region in dependence on the projected driver attention region of the driver. Advantageously computation is reduced by determining the one of the plurality of predetermined driver attention regions. Optionally, determining whether any of the one or more objects falls outside of the projected driver attention region of the driver comprises determining that a first one of the any of the one or more objects falls outside of the projected driver attention region in dependence on a path from the location of the head of the driver to the location of the first object intersecting another driver attention region. Advantageously, there is provided a way for determining whether an object is outside the projected driver attention region of the driver, and therefore whether the driver should be alerted to the presence of the object. Optionally, the current attention region differs from the another driver attention region. The another driver attention region is a different driver attention region from amongst the plurality of driver attention regions to the current driver attention region. The current driver attention region may be a first driver attention region and the another driver attention region is a second driver attention region. Advantageously, this provides an improved process for determining whether an object is outside the projected driver attention region of the driver. Optionally, intersecting is understood to mean that the projected driver attention region passes through or intersects, at least partly, the current driver attention region. Optionally, at least some of the plurality of driver attention regions each correspond to at least a portion of one or more windows of the vehicle. Advantageously, the driver may be alerted to an object that they may not have seen while the driver is looking, i.e. their gaze is directed, through a window of the vehicle. The driver may be alerted to an object that is visible from a window, or region of a window, that is different to a window, or region of a window, through which the driver is currently gazing. The driver may be alerted to an object that is visible from the window through which the driver is currently gazing, but where it is determined that the driver is not gazing (or has not gazed) at the detected object. Optionally, at least some of the plurality of driver attention regions each correspond to a display device. The display device may be arranged in use to display a region of the environment of the vehicle. For example, the display device may provide a rear view of the vehicle. Advantageously, the driver may be alerted to an object that they may not have seen using the display device. Further advantageously, the alert may be provided on the display device to draw the driver’s attention to a detected object. Optionally, the one or more processors are collectively configured to: determine a location upon one more windows of the vehicle at which to generate the alert corresponding to the location of the objects which fall outside of the projected driver attention region of the driver; and output the control signal output to the driver alert system indicative of the location upon the one or more windows of the vehicle at which to display the visual indicator. Optionally, the one or more processors are collectively configured to determine the location upon the one or more windows such that the visual indicator augments the appearance of the any of the one or more objects. Advantageously, the driver’s attention may be drawn directly or more precisely to the detected object rather than a general area on the window or display. This allows the driver to be more quickly aware of the object, so the driver can more quickly make decisions regarding how to control the vehicle in response to viewing the detected object. Optionally, augmenting the appearance comprises the visual indicator substantially following a periphery or shape of the object. Advantageously, the following highlights or emphasizes the detected object whilst remaining relatively unobtrusive. That is, the driver’s attention may be drawn to the detected object without the visual indicator taking up an overly large portion of the window or display. Further advantageously, the driver may more readily ascertain the object that is being augmented. This enables the driver to more quickly act upon the knowledge that the object is present. Optionally, the one or more processors are collectively configured to determine the location upon the one or more windows to substantially align with a path from the location of the head of the driver to the location of the any of the one or more objects falling outside of the projected driver attention region of the driver. Advantageously the alignment provides more accurate positioning of the visual indicator. Optionally the one or more processors are collectively configured to update the location upon the one more windows of the vehicle at which the visual indicator is generated in dependence on a change in the location of the head of the driver. Advantageously the updating the location maintains the alignment. Advantageously, the visual indicator is updated to highlight or emphasize the detected object even when the head of the driver moves. The driver’s attention is readily drawn to the object by the updating. Optionally, the projected driver attention region is a region representing an estimated gaze of the driver in the gaze direction. Advantageously, this enables an estimated gaze of the driver to be determined. The objects that the driver is looking at, or has looked at, can then be determined. Consequently, the objects that the driver has not looked at can then be determined. According to another aspect of the invention, there is provided a system. The system comprises the control system and a driver alert system arranged to provide an alert corresponding to a location of one or more objects in an environment external to the vehicle. Optionally, the driver alert system is arranged to project the visual indicator onto an interior surface of the vehicle corresponding to the location of one or more objects in an environment of the vehicle. Advantageously, the driver is provided with a clear indication of an object that they may not have seen and the location of said object. Optionally, the interior surface is an interior surface of at least one window of the vehicle or a display device. Advantageously, the driver can readily view the indicator on the interior surface. Optionally, the system comprises a driver monitoring system arranged to provide gaze data indicative of a direction of gaze of a driver of the vehicle to the control system. Advantageously, this enables the one or more objects that the driver may not have looked at to be determined. Optionally, the system comprises a sensor system configured to provide, to the control system, object data indicative of a location of one or more objects in an environment external to the vehicle. Advantageously, the objects external to the vehicle can be determined. The locations of these objects can also be determined. According to yet another aspect of the invention, there is provided a vehicle. The vehicle comprises a control system or a system. According to a further aspect of the invention, there is provided a method for controlling a driver alert system of a vehicle. The method comprises receiving object data indicative of a location of one or more objects in an environment external to the vehicle; receiving gaze data indicative of a direction of gaze of a driver of the vehicle; determining, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of a driver; and outputting a control signal to the driver alert system of the vehicle to generate an alert if any of the one or more objects falls outside of the projected driver attention region of the driver. Optionally, the alert is a visual indicator corresponding to the location of the one or more objects. Optionally, the method further comprises determining one of a plurality of predetermined driver attention regions as a current driver attention region in dependence on the projected driver attention region of the driver, and determining that a first one of the any of the one or more objects falls outside of the projected driver attention region in dependence on a path from the location of a head of the driver to the location of the first object intersecting another driver attention region. Optionally, the method further comprises determining a location upon one more windows of the vehicle at which to generate the visual indicator corresponding to the location of the any of the one or more objects falling outside of the projected driver attention region of the drive; and outputting the control signal output to the driver alert system indicative of the location upon the one or more windows of the vehicle at which to display the visual indicator. According to another aspect of the invention, there are provided computer readable instructions which, when executed by one or more processors, cause the one or more processors to perform the method for controlling a driver alert system as described herein. Within the scope of this application it is expressly intended that the various aspects, embodiments, examples and alternatives set out in the preceding paragraphs, in the claims and / or in the following description and drawings, and in particular the individual features thereof, may be taken independently or in any combination. That is, all embodiments and / or features of any embodiment can be combined in anyway and / or combination, unless such features are incompatible. The applicant reserves the right to change any originally filed claim or file any new claim accordingly, including the right to amend any originally filed claim to depend from and / or incorporate any feature of any other claim although not originally claimed in that manner. BRIEF DESCRIPTION OF THE DRAWINGS One or more embodiments of the invention will now be described, by way of example only, with reference to the accompanying drawings, in which: Figure 1 shows a vehicle in accordance with examples of the invention; Figure 2 shows a first block diagram illustrating a control system in accordance with examples of the invention; Figure 3 shows a second block diagram illustrating a system in accordance with examples of the invention; Figure 4A shows a gaze area in accordance with examples of the invention; Figure 4B shows another example of a gaze area in accordance with examples of the invention; Figure 5A shows driver attention regions on a windscreen in accordance with examples of the invention; Figure 5B shows an example of an alert in accordance with examples of the invention; Figure 5C shows an example of determining a location of an alert in accordance with examples of the invention; Figure 6A shows another example of an alert in accordance with examples the invention; Figure 6B shows another example of an alert in accordance with examples of the invention; Figure 6C shows another example of determining a location of an alert in accordance with examples of the invention; and Figure 7 is a flow chart showing a method according to examples of the invention. DETAILED DESCRIPTION A vehicle 10 in accordance with an embodiment of the present invention is described herein with reference to the accompanying Figure 1. The vehicle 10 is a land-going vehicle 10 comprising a plurality of wheels, although other types of vehicle such as watercraft and aircraft may be envisaged. With reference to Figure 2, there is illustrated a control system 20 for a vehicle 10, such as the vehicle 10. The control system 20 comprises one or more electronic controllers 200. The control system 20 as illustrated in Figure 2 comprises one controller 200, although it will be appreciated that this is merely illustrative and one or more controllers may be envisaged. The controller 200 comprises processing means 210 and memory means 220. The processing means 210 may be one or more electronic processing device 210 which operably executes computer-readable instructions. The memory means 220 may be one or more memory device 220. The memory means 220 is electrically coupled to the processing means 210. The memory means 220 is configured to store instructions, and the processing means 210 is configured to access the memory means 220 and execute the instructions stored thereon. The controller 200 comprises an input means 230 and an output means 240. The input means 230 may comprise an electrical input 230 of the controller 200. The output means 240 may comprise an electrical output 240 of the controller 200. The input 230 is arranged to receive an object detection signal 235 from an object detection sensor. The object detection signal 235 is an electrical signal which is indicative of a location of an object external to the vehicle. The input 230 is further arranged to receive a gaze signal 235 from a gaze sensor or gaze monitor which may comprise one or more camera. The gaze signal 235 is an electrical signal indicative of a direction of a gaze of a driver of the vehicle. The object detection signal and the gaze signal 235 may be present in a combined electrical signal 235 such as representing data on a communication bus of the vehicle 10. The output 240 is arranged to output an indicator control signal 245 for controlling an output of an indicator in response to determining that the driver is not viewing or gazing at the object as will be explained later. Figure 3 illustrates a system 300 comprising the control system 20, as described with reference to Figure 2, and a driver alert system 330. . The driver alert system 330 is arranged to provide an alert for a driver of the vehicle 10, the alert corresponding to a location of one or more objects in an environment external to the vehicle 100. The system 300 comprises a monitoring system 310 which may comprise one or both of a cabin monitoring system (CMS) and a driver monitoring system (DMS) 310. The CMS may comprise one or more interior-facing cameras arranged to provide image data corresponding to at least a portion of the interior of the vehicle 10 which may comprise a view of the driver’s head and / or face. The DMS may comprise at least one camera arranged to face the driver and to provide image data corresponding to at least a portion of the driver’s head and / or face. In dependence on the image data a gaze direction of the driver may be determined. The monitoring system 310 is arranged to output gaze data 315 indicative of a direction of gaze i.e. gaze direction of a driver of the vehicle 10 to the control system 20. The driver monitoring system may be the gaze sensor described in relation to Figure 2. The gaze of the driver is a direction in which the driver is looking. The term gaze does not imply any length of time for which the driver is looking in the gaze direction and the gaze direction may be regularly or frequently updated e.g. at periods of milliseconds, for example. The system 300 comprises a sensor system 320 which is arranged to output object data 325 indicative of a location of one or more objects in the environment external to the vehicle 10 to the control system 20. The sensor system 320 may be the object detection sensor described in relation to Figure 2. The driver alert system 330 receives an alert signal 335 from the control system 20 in dependence on gaze data 315 and the object data 325. The alert signal 335 may be the indicator control signal 245 of Figure 2. The driver alert system 330 may output an alert to a driver of the vehicle 10 in response to receiving the alert signal 335. The alert may be an indicator to draw the driver’s attention to the detected object. Figures 4A and 4E3 illustrate a projected driver attention region 430 of the driver of the vehicle 10. Figure 4A is a view from the inside of the vehicle 10, facing outward i.e. from an interior of the vehicle 10. The window 410 may be a windscreen or other window e.g. side window of the vehicle 10. The driver 420 views the outside environment i.e. external to the vehicle through the window 410. The projected driver attention region 430 may be an area of the window 410 through which the driver 420 is looking or gazing, i.e. to which their attention is directed, in the gaze direction of the driver. The projected driver attention region 430 forms where hypothetical gaze lines 440 from the location of the head or more specifically the eyes of the driver 420 in the gaze direction intersect the window 410. A gaze line 440 is a line drawn from an eye to an outside environment in the gaze direction indicative of the direction that the eye is looking or gazing. The gaze line 440 may be referred to as a line of sight, or other suitable terminology. The projected driver attention region 430 is a region which may have a predetermined shape such as circular or oval of predetermined dimensions e.g. diameter to represent an area being viewed or viewable by the driver of the vehicle 10. The features of the projected driver attention region 430 shown in these Figures are examples, and are not intended to limit the size, shape, proportions or any other features of the driver attention region 430. The projected driver attention region 430 may also not be represented on the window 410, and is a theoretical construct to assist with the understanding of the invention. Figure 4B is a side view of the projected driver attention region 430. The driver 420 within the vehicle views the outside environment via the window 410. The gaze direction of the driver 420 may be determined using one or more cameras 450 such as of the CMS and / or DMS. The one or more cameras 450 may monitor one or more of the head and / or the eyes 420 of the driver to determine the gaze direction. The data obtained by the one or more cameras 450 may be used to determine the gaze direction indicative of the direction in which the driver is looking for any length of time, and the projected driver attention region 430 is determined based on the determined gaze direction. The projected driver attention region 430 is a region or area at a plane of the window indicative of the viewed area of the driver to which their attention is assumed to be directed. Whilst Figures 4A and 4B have been drawn using the example of a window, the skilled person would recognise that the same principles apply if the driver was looking at or through another object. For example, a projected driver attention region 430 may be determined as a region on a display at which the driver is looking. Figure 5A shows examples of driver attention regions 510a, 510b, 510c, 51 Od in accordance with an embodiment of the invention. One or more surfaces of the internal environment of the vehicle 10 may correspond to or be divided into a number of driver attention regions 510a, 510b, 510c, 51 Od at or through which a driver may gaze e.g. windows. In the example of Figure 5A, a surface 500 such as a window, a windscreen or a display is divided into a first driver attention region 510a, a second driver attention region 510b, a third driver attention region 510c and a fourth driver attention region 51 Od. The markings on each of the driver attention regions 510a-d are intended to clearly demarcate the driver attention regions for ease of description and, in practice, the markings are not present i.e. the window of the vehicle is transparent and unmarked. The surface 500 may correspond to one driver attention region e.g. a side window driver attention region or be divided into any number of two or more driver attention regions 510a, 510b, 510c, 51 Od. The driver attention regions 510a, 510b, 510c, 51 Od may be identical or non-identical in terms of size or shape. The driver attention regions 510a, 510b, 510c, 51 Od may extend across a plurality of surfaces. For example, a first driver attention region 510a may be provided across two or more different surfaces, such as two or more windows, such as a portion of a passenger side window and a portion of a windscreen of the vehicle 10. Figure 5B shows an example of an indicator in accordance with an embodiment of the invention. There is a surface 500, such as a windscreen. In a driver attention region 510c, an object 520 in the external environment outside of the vehicle 10 is detected. If it is determined that the driver is not looking at the object 520, or in the direction of the driver attention region 510c, an alert may be deployed. The indicator in this embodiment is a visual indicator 530 i.e. to visually indicate the presence of an object external to the vehicle 10. The visual indicator 530 highlights the driver attention region 510c in which the object 520 is located, or would be viewed through by the driver. The visual indicator may be formed by, for example, one or more lasers or projector devices beaming light onto the surface 500 in dependence on a control signal such as the alert signal 335. The alert may comprise an audible indicator which may comprise one or more sounds or noises which may include a spoken indicator which may provide a guide as to the location of the presence of the object external to the vehicle 10 e.g. “left-hand side” as an example. The alert may comprise a haptic alert. Whilst the object 520 in the Figures shows a human, the skilled person would understand that the object 520 could be any type of object external to the vehicle 10. The object 520 may be any type of object external to the vehicle which could collide with the vehicle. The object 520 may be, but is not limited to, a person, an animal, a vehicle, a bike, a tree, a pillar, a post, a fence, or a wall. The visual indicator 530 may comprise one or more lines corresponding to the object 520 such as forming a border around or on a portion of the driver attention region 510c or a visual effect e.g. to highlight the driver attention region 510c. The one or more lines may be static or dynamic, such as flashing, strobing or moving. The one or more lines may be solid or dashed or otherwise fragmented. The one or more lines may be coloured, such as red. The visual indicator 530 may comprise an area within the driver attention region 510c being tinted or coloured or flashed. That is, a whole ora portion of the driver attention region 510c may be coloured in a colour different to the portions of the surface 500 which do not correspond to i.e. are outside of the driver attention region 510c. Figure 5C shows an example of determining whether to provide the alert according to an embodiment of the invention. The driver of the vehicle gazes in the direction of the gaze lines 440. One or more cameras in the vehicle 10 such as part of the DMS 310 determine the gaze direction that the driver gazes in by monitoring one or both of a head and / or the eyes of the driver. The gaze lines 440 corresponding to the gaze direction of the driver are determined to be passing through the fourth driver attention region 51 Od. The driver attention region through which the gaze lines 440 pass or intersect may be referred to as a current driver attention region i.e. region 51 Od. In the example of Figure 5C, the fourth driver attention region 51 Od is the current driver attention region because the projected driver attention region 430 is located in, or corresponds to, the fourth driver attention region 51 Od. An object 520 outside the vehicle 10 is detected by the sensor system 320 and object data 325 indicative of the object’s location is provided to the control system 20. The driver attention region that the object 520 would be viewable through if the driver 420 was gazing at the object 520 is determined as being located within the driver attention region 510c. The driver attention region corresponding to the object 520 may be determined by inferring hypothetical gaze lines 540 from the object 520 to one or more of the location of the head and / or the eyes of the driver 420 i.e. a hypothetical or required gaze direction. If the hypothetical gaze lines 540 pass through the current driver attention region 51 Od at which the driver is currently gazing, then it may be determined that the driver has or is likely to have seen the object 520. When the hypothetical gaze lines 540 intersect a different driver attention region 510c to the current driver attention region 51 Od, then it may be determined that the driver has not or may not have seen the object 520. In this way it is determined that the location of the object 520 falls outside of the projected driver attention region 430 corresponding to the gaze lines 440 in the direction of gaze of the driver and may not have been seen by the driver. An alert may be consequently output to alert the driver to the object 520. In the example of Figure 5C, the hypothetical gaze lines 540 intersect the window 500 at the third driver attention region 510c, whereas the driver is gazing at the fourth driver attention region 51 Od, so it may be determined that the driver may not have seen the object 520 and an alert 530 may consequently be output at the location of the third driver attention region to draw the driver’s attention toward the object 520. The alert 530 is output by the control system 20 providing the alert signal 335 to the driver alert system 330 indicative of the location at which to generate the alert on the surface or window 500. The alert 530 may prompt the driver to gaze through the driver attention region 510c in which the object 520 is viewable. The one or more cameras 310 in the vehicle may monitorthe gaze of the driver420 as the driver’s attention is changed from one driver attention region to another i.e. from the fourth driver attention region 51 Od to the third driver attention region 510c. Once it is determined that the driver has viewed the object 520, for example because the changed gaze direction of the driver intersects the driver attention region 510c in which the object 520 is viewable, a signal 335 may be output by the control system to stop the indicator 530. In the example of Figure 5C, once the gaze lines 440 corresponding to the direction of gaze of the driver intersect the third driver attention region 510c, it may be determined that the driver has viewed the object 520 and the indicator 530 may stop or cease being output. Once it has been determined that the driver has viewed the object 520, the indicator 530 may not be output again in relation to that object 520, even if the driver’s gaze reverts back to the original driver attention region 51 Od or a different driver attention region 510a, 510b from among the plurality of driver attention regions 510a, 510b, 510c, 51 Od. The location or position of the indicator 530 may dynamically change in dependence on a change in position of one or more of the object 520 (which may be relative to the location of the vehicle) or the head of the driver 420. For example, if the location of the object 520 moves or changes such that the hypothetical lines 540 intersect the surface 510 at a different driver attention region, then the indicator 530 may move to highlight the different region. As another example, if the position of the driver’s head / eyes 420 moves such that the hypothetical lines 540 from the object 520 intersect the surface 500 at a different region, then the indicator 530 may again move to highlight the different region. Figure 6A shows another embodiment of the invention. The alert 610 is a visual indicator provided on a surface 500 such as a window or windscreen or display of the vehicle 10. The visual indicator 610 highlights the object 520 that the driver is deemed not to have, or may not have, seen. One or both of a location or shape of the indicator may correspond to the object 520. For example, the visual indicator 610 may form a border around the object 520. The border may be a square border which may fit tight to the object 520 to minimise space between the object 520 and the border. The border may be a shaped border which follows an outline of the object 520. In either case, the visual indicator substantially may follow a periphery of the object 520. Figures 6E3 and 6C show the positioning of the indicator 610 according to an embodiment of the invention. The driver 420 views the object 520 through the surface 500, such as a windscreen of the vehicle 10. The visual indicator 610 is positioned on or located about the surface 500 such that, if the driver was looking directly at the object 520, the appearance of the object 520 would be augmented by the indicator 610. The indicator 610 is positioned on the surface 500 at the location where gaze lines 440 of the driver 420 in the gaze direction intersect the surface 500. The indicator 610 is output if it is determined that the driver 420 has not gazed, or is not gazing at, the object 520. One or more of the driver’s head 420 and / or eyes may be monitored to determine the gaze direction in which the driver is gazing. One or more of the driver’s head and / or eyes may be monitored by the DMS 310 to determine a portion of the surface 500 through which the driver 420 is gazing, such as a projected driver attention region 430 and / or a driver attention region 51 Oa-d and to provide the system with gaze data indicative thereof. When an object 520 is detected by the object detection system 320 of the vehicle 10, a hypothetical gaze line between the object 520 and one or more of the eyes and / or head of the driver 420 is determined by the control system 20. In one embodiment, the location of the object 520 falling outside of the projected driver attention region 430 of the driver is determined by comparing the hypothetical gaze line with the actual gaze direction of the driver. If an angle between the hypothetical line and the actual gaze direction in which the driver is gazing exceeds a predetermined angle, then it may be determined that the driver has not viewed the object 520. Alternatively the projected driver attention region of the driver may be determined at the surface 500 i.e. at a plane of the windscreen. If the hypothetical gaze line intersects the surface outside of the projected driver attention region, then the location of the object 520 is determined to fall outside of the projected driver attention region 430. The control system 20 may determine a portion of the surface 500 at which the hypothetical line intersects as an object gaze location. The object gaze location may be compared with the projected driver attention region 430. If the object gaze location falls outside of the projected driver attention region 430 then it may be determined that the driver has not viewed the object. If they are separated by at least a predetermined distance, then it may be determined that the driver has not viewed the object 520. When it is determined that the driver has not viewed the object, the indicator 610 may be output to highlight the object 520. Once it has been determined that the driver has viewed the object 520, the indicator 610 may not be output again in relation to that object 520, even if the gaze of the driver is no longer directed to that object 520. Displaying an indicator 610 which substantially follows a periphery of the object 520 may be achieved by triangulation using data from two or more cameras. Monitoring the eyes and / or head of the driver 420 using two or more cameras located at different positions allows the gaze of the driver to be determined more accurately. This then enables a position of the indicator 610 to be more accurately determined. The position of the indicator 610 may be changed if the position of the object 520 changes so that the indicator 610 still follows a periphery of the object. Moreover, the position of the indicator 610 may be changed if a position of the driver’s head changes, so that the indicator 610 still follows the periphery of the object 520 from the driver’s perspective. That is, the position or location of the indicator 610 may be dynamically moved in dependence on one or both of the position of the object (which may be relative to the vehicle) and the position of the driver’s head and / or eyes. Figure 7 illustrates a method 700 according to an embodiment of the invention. The method 700 is a method of controlling a driver alert system 330 of a vehicle 10, such as the vehicle 10 illustrated in Figure 1. In particular, the method 700 is a method of controlling a driver alert system 330 to output analert. The method 700 may be performed by the driver alert system 330 illustrated in Figure 2. In particular, the memory 220 of the controller 20 may comprise computer-readable instructions which, when executed by the processor 210, cause the processor 210 to perform the method 700 according to an embodiment of the invention. In step 710, object data 325 indicative of a location of one or more objects 520 in an environment external to the vehicle 10 is received. The object data 325 may be received by a processor 210 of the control system 20. The object data 325 may be obtained from one or more sensors 320 of the vehicle 10. The object data 325 may be received from one or more sensors 320 associated with the vehicle 10, such as from sensors of a driver assistance system of the vehicle. The one or more sensors may comprise one or more cameras, radar, lidar etc associated with the vehicle 10. The object data 325 may be processed to determine a position and / or location of an object and / or a distance of the object from the vehicle 10. In step 720, gaze data 315 indicative of a direction of gaze of a driver of the vehicle is received. The gaze data may be received by the processor 210. The gaze data 315 may be obtained by one or more cameras, such as the cameras of the CMS and / or DMS. In step 730, it is determined, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of the driver. The determination may be performed by the processor 210. The determination may be performed by comparing a location of the object with the projected driver attention region 430 of the driver as explained above. The determination may be performed by determining a current driver attention region 51 Od based on the received gaze data. The current driver attention region 51 Od is the portion of the internal surface of the vehicle that the driver is currently looking at or through, such as a window or a display. If it is determined that any of the detected objects falls outside the projected driver attention region 430 based on a path of the gaze lines 440 from one or more of the driver’s head and / or eyes intersecting a driver attention region different to the current driver attention region, then it may be determined that the object falls outside a projected driver attention region. It may then be determined that the driver has not viewed the object because it falls outside the projected driver attention region and / or the current driver attention region. The projected driver attention region is a region on a surface representing an estimated gaze of the driver in the direction of gaze of the driver, and is determined using the gaze data received from the one or more sensors. In step 740, a control signal 335 is output to the driver alert system 330 of the vehicle to generate an alert if any of the one or more objects falls outside of the projected driver attention region of the driver. The processor 210 may control the output 240 of the control system 20 to output the control signal 335 to the driver alert system 330. In response to receiving the control signal 335, the driver alert system 330 may generate and output an alert, such as a visual indicator. The alert may augment the appearance of the driver attention region in which the object is viewable. The alert may augment the appearance of the object by providing an indication which substantially follows a periphery of the object. The alert may change location in dependence on a change in position and / or location of one or more of the object and / or the driver’s head. The alert may be any type of indicator to alert a driver to a presence of an object 520 external to the vehicle. For example, the alert may be an audible indicator which sounds an alert. The audible indicator may audibly provide an information relating to a location of the object external to the vehicle. The alert may be a haptic indicator, where a position of the haptic feedback corresponds to a location of the object external to the vehicle. It will be appreciated that various changes and modifications can be made to the present invention without departing from the scope of the present application.
Claims
1. A control system for controlling a driver alert system of a vehicle, the control system comprising one or more processors collectively configured to:receive object data indicative of a location of one or more objects in an environment external to the vehicle;receive gaze data indicative of a direction of gaze of a driver of the vehicle;determine, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of the driver; and output a control signal to the driver alert system of the vehicle to generate an alert if any of the one or more objects fall outside of the projected driver attention region of the driver.
2. The control system as claimed in claim 1, wherein the alert is a visual indicator corresponding to the location of the one or more objects.
3. The control system as claimed in claim 1 or 2, wherein the gaze data is indicative of a location of a head of the driver of the vehicle, and the projected driver attention region is projected from the location of the head of the driver in the direction of gaze of the driver.
4. The control system as claimed in any preceding claim, wherein the determining whether any of the one or more objects fall outside of the projected driver attention region of the driver comprises:determining one of a plurality of predetermined driver attention regions as a current driver attention region in dependence on the projected driver attention region of the driver; anddetermining that a first one of the any of the one or more objects falls outside of the projected driver attention region in dependence on a path from the location of a head of the driver to the location of the first object intersecting another driver attention region.
5. The control system as claimed in claim 4, wherein at least some of the plurality of driver attention regions each correspond to at least a portion of one or more windows of the vehicle.
6. The control system as claimed in any preceding claim, wherein the one or more processors are collectively configured to:determine a location upon one more windows of the vehicle at which to generate the alert corresponding to the location of the objects which fall outside of the projected driver attention region of the driver; andoutput the control signal output to the driver alert system indicative of the location upon the one or more windows of the vehicle at which to display a visual indicator.
7. The control system as claimed in claim 6, wherein the one or more processors are collectively configured to determine the location upon the one or more windows such that the visual indicator augments the appearance of the any of the one or more objects.
8. The control system as claimed in claim 6 or 7 when dependent through claim 3, wherein the one or more processors are collectively configured to determine the location upon the one or more windows to substantially align with a path from the location of the head of the driverto the location of the any of the one or more objects falling outside of the projected driver attention region of the driver; optionally the one or more processors are collectively configured to update the location upon the one more windows of the vehicle at which the visual indicator is generated in dependence on a change in the location of the head of the driver.
9. The control system as claimed in any preceding claim, wherein the projected driver attention region is a region representing an estimated gaze of the driver in the direction of gaze of the driver.
10. A system comprising the control system of any preceding claim and a driver alert system arranged to provide an indicator corresponding to a location of one or more objects in an environment external to the vehicle.
11. The system as claimed in claim 10, wherein the indicator is a visual indicator and the driver alert system is arranged to project the visual indicator onto an interior surface of the vehicle.
12. A vehicle comprising the control system of any of claims 1 to 9 or the system of claims 10 or 11.
13. A method for controlling a driver alert system of a vehicle, the method comprising:receiving object data indicative of a location of one or more objects in an environment external to the vehicle;receiving gaze data indicative of a direction of gaze of a driver of the vehicle;determining, in dependence on the object data and the gaze data, whether any of the one or more objects falls outside of a projected driver attention region of the driver in the direction of gaze of a driver; and outputting a control signal to the driver alert system of the vehicle to generate an alert if any of the one or more objects falls outside of the projected driver attention region of the driver.
14. The method as claimed in claim 13, wherein the alert is a visual indicator corresponding to the location of the one or more objects.
15. The method as claimed in claim 13 or 14, comprising:determining one of a plurality of predetermined driver attention regions as a current driver attention region in dependence on the projected driver attention region of the driver; anddetermining that a first one of the any of the one or more objects falls outside of the projected driver attention region in dependence on a path from the location of a head of the driverto the location of the first object intersecting another driver attention region.
Citation Information
Patent Citations
ViewJP2018022349AonEspacenetopensinnewtab
ViewUS2018/0362053A1onEspacenetopensinnewtab
ViewUS2016/0288709A1onEspacenetopensinnewtab
ViewUS2010/0253594A1onEspacenetopensinnewtab
ViewJP2010126135AonEspacenetopensinnewtab
