Warning of an impending accident outside the line of sight using reflective head-up displays

Abstract

A system and method for warning of impending out-of-sight collisions using reflective head-up displays comprises receiving sensor data detected by a vehicle's sensor system indicating an object moving toward the vehicle, and determining, based on the sensor data, that the object is outside the driver's line of sight. The system and method also comprise determining that the object's trajectory and the vehicle's trajectory will intersect, and displaying, via head-up displays, a graphical warning message alerting the driver to the object being outside their line of sight.

Description

introduction

[0001] The present invention relates to a computer-implemented method.

[0002] Document DE 10 2023 003 826 A1 describes a computer-implemented method according to the preamble of claim 1. Documents DE 10 2017 106 931 A1 and DE 10 2017 109 513 A1 disclose related methods.

[0003] This disclosure relates generally to a system and method for warning of impending non-light-of-sight (non-light-of-sight) crashes or accidents using reflective head-up displays. Generally, vehicle collisions can be classified into one or more types. For example, collision types can include single-vehicle, reversing, frontal, rear-end, side-impact, and / or angle collisions, among others. While current vehicle sensor systems are particularly adept at identifying and warning of potential collisions within a driver's line of sight (e.g., the driver's gaze while looking forward) using current displays, a large percentage (e.g., 27%) of all annual vehicle collisions involve non-light-of-sight collision types such as side-impact and angle collisions.

[0004] Furthermore, for safety reasons, it is crucial that any visual alerting procedures do not divert the driver's primary attention from the road ahead. For example, providing out-of-sight warnings using pillar-to-pillar display capabilities can integrate safety alerts with the driver's view of the road while minimizing distractions. Additionally, providing the driver with a message about the urgency and direction of an impending collision, which would otherwise be outside the driver's line of sight, gives the driver time to take corrective action to avoid the collision. Summary

[0005] An aspect of the invention provides a computer-implemented method for warning of impending out-of-line accidents using reflective head-up displays, which, when executed on data processing hardware, causes the data processing hardware to perform operations that include receiving sensor data detected by a vehicle's sensor system, wherein the sensor data indicates an object moving towards the vehicle, and determining, based on the sensor data, that the object is outside the driver's line of sight.The operations also include determining that an object's trajectory and a vehicle's trajectory will intersect, and displaying, via head-up displays, a graphical warning message to alert the vehicle's driver to the object, which is outside the driver's line of sight.

[0006] The head-up displays include an augmented reality head-up display and a blackout head-up display. Displaying a graphical warning message via the head-up displays, alerting the driver to an object outside their line of sight, further involves generating an augmented reality image overlay, creating a virtual image, and simultaneously projecting the augmented reality image overlay and the virtual image onto the vehicle's windshield. This projection onto the windshield includes projecting the augmented reality image overlay onto a transparent portion of the windshield and projecting the virtual image onto a blackout portion.

[0007] In addition, the augmented reality image overlay differs from the virtual image.

[0008] In some examples, the sensor system includes one or more cameras, radio detection and ranging (RADAR), and light detection and ranging (LIDAR). In some implementations, determining whether an object's trajectory and a vehicle's trajectory will intersect involves receiving vehicle data, calculating the object's trajectory based on the sensor data, calculating the vehicle's trajectory based on the vehicle data, and determining whether the object's and vehicle's trajectories intersect. In these implementations, the operations may further include determining, based on the object's and vehicle's trajectories, the time until a collision between the object and the vehicle.These operations can further include generating a graphical warning message based on the time until the collision between the object and the vehicle. In some examples, displaying the graphical warning message via the head-up display to alert the driver to the object being outside their line of sight includes displaying the graphical warning message on the vehicle's windshield to indicate the object's direction.

[0009] Another, not independently claimed, aspect of the disclosure provides a system for warning of impending out-of-sight accidents using reflective head-up displays, comprising data processing hardware and storage hardware communicating with the data processing hardware. The storage hardware stores instructions which, when executed by the data processing hardware, cause the data processing hardware to perform operations that include receiving sensor data detected by a vehicle's sensor system, wherein the sensor data indicates an object moving toward the vehicle, and determining, based on the sensor data, that the object is outside the driver's line of sight.The operations also include determining that an object's trajectory and a vehicle's trajectory will intersect, and displaying, via head-up displays, a graphical warning message to alert the vehicle's driver to the object, which is outside the driver's line of sight.

[0010] This aspect can include one or more of the following optional features. In some implementations, the head-up displays include an augmented reality head-up display and a blackout head-up display. In these implementations, displaying a graphical warning message via the head-up displays to alert the driver to an object outside their line of sight can include generating an augmented reality image overlay, generating a virtual image, and simultaneously projecting the augmented reality image overlay and the virtual image onto the vehicle's windshield.Projecting the augmented reality image overlay and the virtual image onto the vehicle's windshield can further include projecting the augmented reality image overlay onto a transparent portion of the windshield and projecting the virtual image onto a blackout portion. Additionally or alternatively, the augmented reality image overlay differs from the virtual image.

[0011] In some examples, the sensor system includes one or more cameras, radio-based detection and distance measurement (RADAR), and light-based detection and distance measurement (LIDAR). In some implementations, determining whether an object's trajectory and a vehicle's trajectory will intersect involves receiving vehicle data, calculating the object's trajectory based on the sensor data, calculating the vehicle's trajectory based on the vehicle data, and determining whether the object's and vehicle's trajectories intersect. In these implementations, the operations may further include determining, based on the object's and vehicle's trajectories, the time until a collision between the object and the vehicle.These operations can further include generating the graphical warning message based on the time until the collision between the object and the vehicle. In some examples, displaying the graphical warning message via the head-up display to alert the driver to the object being outside their line of sight includes displaying the graphical warning message on the vehicle's windshield to indicate the object's direction.

[0012] The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the description below. Further aspects, features, and advantages will become apparent from the description, the drawings, and the claims. Brief description of the drawings

[0013] The drawings described herein serve only to illustrate selected configurations. Fig. Figure 1 is a schematic view of an exemplary system that uses reflective head-up displays to warn of an impending accident outside the line of sight. Fig. Figure 2 is a schematic view of exemplary components of the system of Fig. 1. Fig. Figure 3 is a flowchart of a crash or accident model for the system of Fig. 1. Fig. Figure 4 is a schematic view of the system's head-up displays. Fig. 1. Fig. Figure 5 is a schematic view of the system's head-up displays. Fig. 1. Fig. Figure 6 is a flowchart of an exemplary arrangement of operations for a procedure for warning of impending accidents outside the line of sight using reflective head-up displays.

[0014] The corresponding reference numbers identify the corresponding parts in the drawings. Detailed description

[0015] With reference to the accompanying drawings, exemplary configurations are now described in more detail. These exemplary configurations are provided in such a way as to make this disclosure complete and to fully convey the scope of the disclosure to the person skilled in the art. Specific details are set forth, such as examples of certain components, devices, and methods, to enable a comprehensive understanding of the configurations of this disclosure. It is obvious to the person skilled in the art that specific details need not be used and that exemplary configurations can be embodied in many different forms.

[0016] The terminology used herein serves only to describe certain exemplary configurations and is not to be understood as restrictive. As used herein, the articles "a," "an," and "the" in the singular are intended to include the plural forms unless the context clearly indicates otherwise. The terms "comprise," "comprehensive," "inclusive," and "exhibiting" are integrative and thus specify the presence of features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The procedural steps, processes, and operations described herein are not to be interpreted as requiring that they be carried out in the order discussed or illustrated, unless they are expressly identified as the order of implementation.Additional or alternative steps can be used.

[0017] When an element or layer is referred to as being "on," "interacting with," "connected with," "attached to," or "coupled with" another element or layer, it may be directly on, interacting with, connected with, attached to, or coupled with the other element or layer, or there may be intervening elements or layers. Conversely, when an element is referred to as being "directly on," "directly interacting with," "directly connected with," "directly attached to," or "directly coupled with" another element or layer, there may be no intervening elements or layers. Other words used to describe the relationship between elements should be interpreted similarly (e.g., "between" versus "directly between," "adjacent" versus "directly adjacent," etc.).As used herein, the term “and / or” includes all combinations of one or more of the associated listed elements.

[0018] The terms "first," "second," "third," etc., may be used herein to describe different elements, components, areas, layers, and / or sections. These elements, components, areas, layers, and / or sections are not intended to be restricted by these terms. These terms may only be used to distinguish one element, component, area, layer, or section from another. Terms such as "first," "second," and other numerical terms do not imply any sequence or order unless clearly evident from the context. Thus, a first element, first component, first area, first layer, or first section could be...A first section, which is referred to below, may also be called the second element, second component, second area, second layer or second section, without deviating from the teachings of the exemplary configurations.

[0019] In this application, including the definitions below, the term "module" may be replaced by the term "circuit". The term "module" may refer to, be part of, or comprise an application-specific integrated circuit (ASIC), a digital, analog, or mixed analog / digital discrete circuit, a digital, analog, or mixed analog / digital integrated circuit, a combinational logic circuit, a field-programmable gate array (FPGA), a processor (common, dedicated, or group) that executes code, a memory (common, dedicated, or group) that stores the code executed by a processor, other suitable hardware components that provide the described functionality, or a combination of some or all of the above, such as in a system-on-a-chip.

[0020] The term "code," as used above, can include software, firmware, and / or microcode, and can refer to programs, routines, functions, classes, and / or objects. The term "shared processor" includes a single processor that executes part or all of the code of multiple modules. The term "group processor" includes a processor that, in combination with additional processors, executes part or all of the code of one or more modules. The term "shared memory" includes a single memory that stores part or all of the code of multiple modules. The term "group memory" includes memory that, in combination with additional memory, stores part or all of the code of one or more modules. The term "memory" can be a subset of the term "computer-readable medium."The term "computer-readable medium" does not include the transient electrical and electromagnetic signals that propagate through a medium and can therefore be considered a concrete, or physical, and non-transient storage medium. Non-restrictive examples of non-transient storage include physical computer-readable media, including non-volatile memory, magnetic memory, and optical memory.

[0021] The devices and methods described in this application can be partially or completely implemented by means of one or more computer programs executed by one or more processors. The computer programs contain instructions executable by processors, which are stored on at least one non-transient, physical, computer-readable medium. The computer programs can also contain and / or access stored data.

[0022] A software application (i.e., a software resource) can refer to computer software that causes a computer device to perform a task. In some examples, a software application may be referred to as an "application," "app," or "program." Examples of applications include, but are not limited to, system diagnostic applications, system administration applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.

[0023] Non-transitory memory can be physical devices used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program status information) for use by a computing device. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only or read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (which is typically used, for example, for firmware such as boot programs). Examples of volatile memory include, but are not limited to, random access memory.Random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and floppy disks or tapes.

[0024] These computer programs (also referred to as programs, software, software applications, or code) contain machine instructions for a programmable processor and can be implemented in a procedural high-level and / or object-oriented programming language and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, device, and / or equipment (e.g., magnetic disks, optical disks, memory, programmable logic devices (PLDs)) used to provide machine instructions and / or data to a programmable processor containing a machine-readable medium that receives machine instructions as a machine-readable signal.The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0025] Various implementations of the systems and techniques described herein may be realized in a digital electronic and / or optical circuit arrangement, an integrated circuit arrangement, specially designed or engineered ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include an implementation in one or more computer programs that are executable and / or interpretable on a programmable system containing at least one programmable processor, which may be designed or suited for special or general purposes and is coupled such that it receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits data and instructions to these.

[0026] The processes and logical sequences described in this description can be executed by one or more programmable processors, also referred to as data processing hardware, which execute one or more computer programs to perform functions by processing input data and producing an output. The processes and logical sequences can also be executed by a special-purpose logic circuit arrangement, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). Processors suitable for executing a computer program include, for example, general-purpose and specialized microprocessors, as well as any or multiple processors of any type of digital computer. Generally, a processor receives instructions, or...Instructions and data from read-only memory or random-access memory, or both. The essential elements of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Generally, a computer also includes one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or is functionally coupled to them to receive data from or to them, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and storage devices, including, for example, semiconductor storage devices such as EPROM, EEPROM, and flash memory devices, and magnetic disks, such as...Internal hard drives or removable disks, magneto-optical disks, as well as CD-ROM and DVD-ROM disks. The processor and memory can be supplemented by or integrated into a logic circuit for special purposes.

[0027] To enable user interaction, one or more aspects of the disclosure can be implemented on a computer that includes a display device, such as a CRT (cathode ray tube), LCD (liquid crystal display), or touchscreen monitor, for displaying information to the user, and optionally a keyboard and pointing device, such as a mouse or trackball, with which the user can input information into the computer. Other types of devices can also be used to enable user interaction; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including acoustic, verbal, or tactile input.Furthermore, a computer can interact with a user by sending documents to and receiving documents from a device used by the user, for example by sending web pages to a web browser on a user's client device in response to requests received by the web browser.

[0028] Referring to Fig. In some implementations, 1 comprises a system 100, a vehicle 10, and / or a remote system 60, which communicates with the vehicle 10 via a network 40. The vehicle 10 and / or the remote system 60 execute a system 200 for a warning of an impending out-of-sight accident (also referred to as accident warning system 200). Fig. 2) In short, and as described in more detail below, the collision warning system 200 is configured to receive sensor data 20 indicating that an object 30 is moving towards the vehicle 10, and, if the object 30 is outside the driver's line of sight 104 of the vehicle 10, to display a graphical warning message 202 via head-up displays 204, alerting the driver 102 to the object 30. In particular, by alerting the driver 102 to the object 30 being outside the driver's line of sight 104, the driver 102 is given time to take corrective action to avoid imminent collision hazards that would otherwise not be visible to the driver 102.The imminent accident hazards, which would otherwise not be visible to the driver, include, for example, an accident at an intersection because another driver runs a red light, or a side collision when another driver passes vehicle 10.

[0029] As used herein, an object 30 located outside the driver's line of sight 104 can generally refer to the positioning of the object 30 relative to the vehicle 10 in real time such that a vehicle occupant (e.g., the driver 102) cannot perceive the object 30 when looking towards the front of the vehicle 10. The perception of the object 30 can be based, at least in part, on where the vehicle occupant is seated within the vehicle 10 and includes areas outside the vehicle 10 that are not naturally observable when the vehicle occupant's head is turned towards the front of the vehicle 10. These areas may also include areas outside the vehicle 10 that are not naturally observable when the vehicle occupant's head is turned from the neck to the right and to the left. An example of the driver's line of sight 104 is given in Fig. Figure 2 shows where the dashed arrows define the line of sight 104 and where objects 30 located within the line of sight 104 can be perceived by the driver 102 when looking towards the front of the vehicle 10. In some implementations, the line of sight 104 is a conical area in the direction of travel of the vehicle 10 in front of the vehicle with a field of view of 120 degrees. The line of sight 104 can also extend over a distance of approximately 800 meters from the driver 102. In other implementations, the line of sight 104 is dynamic based on the geographical area and weather conditions in which the vehicle 10 is traveling.

[0030] In the examples shown, the accident warning system 200 is implemented in a vehicle 10. However, the accident warning system 200 can also be implemented on other computing devices (e.g., computing devices connected to the vehicle 10), such as, without limitation, a smartphone, a tablet, a smart display, a desktop / laptop, a smartwatch, a smart device, or smart glasses / headset. The vehicle 10 includes data processing hardware 12 and storage hardware 14, which stores instructions that, when executed on the data processing hardware 12, cause the data processing hardware 12 to perform operations. The vehicle 10 also includes a sensor system 16 configured to acquire / receive sensor data 20.The sensor system 16 can include one or more cameras, a radio-based detection and distance measurement system (RADAR), and a light-based detection and distance measurement system (LIDAR) capable of capturing image data. While this is in . Fig. Although the sensor system 16 shown in Figure 1 is located on the front of the vehicle, it should be noted that the sensor system 16 can include sensors located anywhere in the vehicle. For example, the sensor system 16 can perform 360-degree environmental sensing of the vehicle's surroundings 10.

[0031] The remote system 60 (e.g., server, cloud computing environment) also includes data processing hardware 62 and storage hardware 64, which stores instructions that, when executed on the data processing hardware 62, cause the data processing hardware 62 to perform operations. In some examples, the execution of the accident warning system 200 is split between the vehicle 10 and the remote system 60. As referred to in Fig. 2 and Fig. As described in more detail below, the crash warning system 200, which is executed on the vehicle 10 and / or the remote system 60, performs a crash or accident model 300, which is configured to receive the sensor data 20 detected by means of the sensor system 16 and to generate the graphical warning message 202 when the sensor data 22 indicate that there is an imminent risk of an accident with the object 30 for the vehicle 10 and that the object 30 is outside the line of sight 104 of the driver 102 of the vehicle 10.

[0032] As in Fig. 1 and Fig. As shown in Figure 2, the vehicle 10 also includes a windshield 18, which provides the crash warning system 200 with pillar-to-pillar display options. Specifically, the windshield 18 includes a transparent portion 24 and a blackout portion 26. The transparent portion 24 can generally refer to the part of the windshield 18 through which the driver 102 perceives areas outside the vehicle 10. The blackout portion 26 can generally refer to an opaque or darkened area of ​​the windshield 18 in which, for example, a vacuum fluorescence display (VFP), a light-emitting diode (LED) display, a driver information center display, a radio display, any text device, a head-up display (HUD), a touchscreen display, a liquid crystal display (LCD), etc., is displayed.An instrument cluster and / or an infotainment system may be displayed.

[0033] Referring to Fig. 1-3 While the vehicle 10 is in motion, it executes the accident warning model 300, which receives as input the sensor data 20 detected by the vehicle 10's sensor system 16. The sensor data 20 can comprise one or more data fragments or image data detected by the sensor system 16 and can indicate that an object 30 is moving in the direction of the vehicle 10. For example, the object 30 could be another vehicle located outside the driver's line of sight 104 of the vehicle 10. However, the object 30 could be any object capable of causing a collision with the vehicle 10, such as, without limitation, motorcycles, trucks, SUVs (sport utility vehicles), recreational vehicles, off-roaders, etc.The accident warning system 200 can additionally receive vehicle data 22, including the direction of the vehicle 10, the speed of the vehicle 10, and / or the current location of the vehicle 10. The accident warning model 300 then determines whether there is an imminent risk of an accident between the vehicle 10 and the object 30 and generates the graphical warning message 202 for output to the head-up displays 204.

[0034] Referring to Fig. Figure 3 depicts the accident warning model 300. Here, while the vehicle 10 is in motion, the accident warning model 300 continuously receives and processes sensor data 20 from sensor system 16 and vehicle data 22 to determine whether to output the graphical warning message 202 to the head-up displays 204. In operation 310, the accident warning model 300 receives sensor data 20 detected by sensor system 16 of the vehicle 10 and vehicle data 22 of the vehicle 10. In operation 320, the accident warning model 300 then determines whether any objects 30 are detected moving towards the vehicle 10. For example, based on the location of the vehicle 10 and the location of the objects 30, the accident warning model 300 can determine whether any sensor data 20 contains objects 30 approaching the vehicle 10.If the accident warning model 300 determines that an object 30 is moving towards the vehicle 10, the operations further include, in operation 330, calculating a trajectory of the object 30. Here, the trajectory of the object 30 can refer to a position, direction and / or speed of the object 30 based on the current sensor data 20 and / or recently received sensor data 20.

[0035] In operation 340, the collision warning model 300 calculates a trajectory of vehicle 10. For example, based on vehicle data 22, the collision warning model 300 can determine a position, direction, and / or speed of vehicle 10, where the trajectory of vehicle 10 can refer to the position, direction, and / or speed of vehicle 10. In some implementations, the vehicle data 22 is measured / reported using an inertial measurement unit (IMU). After calculating the trajectory of object 30 and the trajectory of vehicle 10, the collision warning model 300 determines in operation 350 whether the trajectory of object 30 and the trajectory of vehicle 10 intersect.In other words, the collision warning model 300 determines, based on the respective trajectories of object 30 and vehicle 10, whether a collision between object 30 and vehicle 10 is imminent. In doing so, the collision warning model 300 can compare the position, direction, and / or speed of object 30 with the position, direction, and / or speed of vehicle 10 and, if the trajectories indicate that object 30 and vehicle 10 will collide, proceed to operation 360.

[0036] In Operation 360, the collision warning model 300 determines how soon the trajectories of object 30 and vehicle 10 will intersect. Specifically, based on the trajectories of object 30 and vehicle 10, collision warning model 300 determines a time until collision between object 30 and vehicle 10. In these implementations, the graphical warning message 202 can be generated based on this time until collision. In other words, the graphical warning message 202 can be configured based on the urgency indicated by the time until collision between object 30 and vehicle 10. Specifically, the size, highlighting, colors, and / or flashing of the graphical warning message 202 can change depending on the urgency indicated by the time until collision between object 30 and vehicle 10.For example, if the time until the collision between object 30 and vehicle 10 is longer (e.g., greater than five (5) seconds) and therefore less urgent, the collision warning model 300 can, during operation 370, generate / display a graphical warning message 202 warning driver 102 of object 30 being outside driver 102's line of sight 104. In this case, the graphical warning message 202 warning driver 102 can use smaller graphics, colors, and / or flashing elements to indicate that the approaching object 30 is less urgent. If the time until the collision between object 30 and vehicle 10 is shorter (e.g., less than five (5) seconds) and therefore urgent, the collision warning model 300 can, during operation 380, generate / display a graphical warning message 202 warning driver 102 of object 30 being approaching outside driver 102's line of sight 104.In this context, the graphical warning message 202, which warns the driver 102, may use larger graphics, brighter / stronger colors and / or flashing elements to indicate that the approaching object 30 is urgent and likely to result in a collision.

[0037] Back on Fig. Referring to 2, after the accident warning model 300 determines that object 30 and vehicle 10 are on trajectories that will intersect, the accident warning system 200 generates the graphical warning message 202. Specifically, the accident warning system 200 outputs the graphical warning message 202 to the head-up displays 204. As shown, the head-up displays 204 comprise an augmented reality head-up display 206 and a blackout head-up display 208, wherein the augmented reality head-up display 206 is configured to project images onto the transparent portion 24 of the windshield 18, and the blackout head-up display is configured to project images onto the blackout portion 26 of the windshield 18. The graphical warning message 202 can generally contain an augmented reality image overlay 210 and a virtual image 212, each projected by the respective components of the head-up displays 204.For example, the augmented reality head-up display 206 can receive the graphical warning message 202, which contains the augmented reality image overlay 210, and project the augmented reality image overlay 210 onto the transparent portion 24 of the windshield at a location on the windshield 18 within the driver's line of sight 104. In some implementations, the augmented reality image overlay 210 indicates a direction in which the object 30, which is located outside the driver's line of sight 104, is moving. Similarly, the blackout head-up display 208 can be configured to display images on the blackout portion 26 of the windshield 18. In particular, the blackout head-up display 208 can receive the graphic warning message 202 containing the virtual image 212 and project the virtual image 212 onto the blackout part 26 of the windscreen 18.In some implementations, the augmented reality image overlay 210 and the virtual image 212 are projected simultaneously onto the windscreen 18.

[0038] Referring to Fig. 4 and Fig. Figure 5 shows exemplary components 400 and 500, which include the vehicle 10, which executes the collision warning system 200 to warn the driver 102 that an object 30 outside the driver's line of sight 104 poses a risk of imminent collision with the vehicle 10. By warning the driver 102 (e.g., via the graphical warning message 202), the driver 102 is given additional time to react / avoid the collision. It should be noted that the graphical warning message 202 projected onto the windshield 18 can be configured / modified depending on the urgency of the time until the collision between the object 30 and the vehicle 10. For example, the graphical warning message 202 can use images of different sizes, colors and / or other warning techniques to attract the attention of the driver 102 without distracting the driver 102 from the road.In some implementations, the augmented reality image overlay 210 differs from the virtual image 212.

[0039] With reference to Fig. 4. Based on sensor data 20, the collision warning system 200 can detect that an object 30 (e.g., another vehicle, not shown) to the left of vehicle 10 is likely to collide with vehicle 10. For example, object 30 may be on a trajectory that includes running a red light, so that it will collide with vehicle 10 at the intersection. In this case, object 30 is outside the driver's line of sight 104, so the driver cannot see object 30. In response to the detection of object 30 and the likelihood of it colliding with vehicle 10, the collision warning system 200 generates the graphical warning message 202, including the augmented reality image overlay 210a and the virtual image 212a.As shown, the augmented reality image overlay 210a is projected onto the transparent part 24 of the windshield 18 at a point on the windshield 18 within the driver's line of sight 104, so that the augmented reality image overlay 210a appears to be positioned on the road in front of the vehicle 10. Here, the augmented reality image overlay 210a contains an image of two vehicles with a crash symbol to alert the driver 102 to the potential collision. Simultaneously, the virtual image 212a is projected onto the blackout part 26 of the windshield 18 on the left side of the windshield 18 to indicate that the object is located to the left of the vehicle 10.

[0040] With reference to Fig. 5. Based on sensor data 20, the accident warning system 200 can detect that an object 30 (e.g., another vehicle) Fig. 1)) to the left of vehicle 10, it is likely that vehicle 10 will collide with it. For example, object 30 may be on a trajectory where it attempts to overtake vehicle 10 in an unsafe manner, which will likely result in a side impact with vehicle 10. In this case, object 30 is outside the driver's line of sight 104, so driver 102 cannot see object 30. In response to the detection of object 30 and the likelihood of it colliding with vehicle 10, the collision warning system 200 generates the graphical warning message 202, including the augmented reality image overlay 210b and the virtual image 212b.As shown, the augmented reality image overlay 210b is projected onto the transparent part 24 of the windshield 18 at a point on the windshield within the driver's line of sight 104, so that the augmented reality image overlay 210a appears to be positioned on the road in front of the vehicle 10. Here, the augmented reality image overlay 210a contains an image of a vehicle on the left side of the windshield 18 with an accident symbol to alert the driver 102 to the potential collision. Simultaneously, the virtual image 212b is projected onto the blackout part 26 of the windshield 18 on the left side and contains an image of a vehicle with a right-pointing arrow to alert the driver 102 that the trajectory of object 30 is to the right, in the direction of vehicle 10.

[0041] Fig. Section 6 includes a flowchart of an exemplary sequence of operations for a Procedure 600 for warning of imminent out-of-sight accidents using reflective head-up displays. The Procedure 600 can be described with reference to Fig. 1-5 are described. A data processing hardware (e.g., the data processing hardware 12, 62 of Fig. 1) can execute instructions stored in memory hardware (e.g., memory hardware 14, 64 of Fig. 1) are stored to perform the exemplary sequence of operations for procedure 600.

[0042] In operation 602, method 600 comprises receiving sensor data 20 detected by a sensor system 16 of a vehicle 10. The sensor data 20 may indicate that an object 30 is moving in the direction of the vehicle 10. In operation 604, method 600 also comprises determining, based on the sensor data 20, that the object 30 is outside the line of sight 104 of a driver 102 of the vehicle 10. In operation 606, method 600 further comprises determining that the trajectory of the object 30 and the trajectory of the vehicle 10 will intersect. The procedure 600 also includes, in operation 608, the display, via head-up displays 204, of a graphic warning message 202, which alerts the driver 102 of the vehicle 10 to the object 30 which is outside the line of sight 104 of the driver 102.

[0043] Several implementations have been described. Nevertheless, it is understood that various modifications can be made without deviating from the spirit and scope of the revelation.

Claims

[1] A computer-implemented method which, when executed on data processing hardware (62), causes the data processing hardware (62) to perform operations, comprising: Receiving (310) sensor data (20) detected by means of a sensor system (16) of a vehicle (10), wherein the sensor data (20) indicate an object (30) moving in the direction of the vehicle (10); Determine (320), based on the sensor data (20), that the object (30) is outside a line of sight (104) of a driver (102) of the vehicle (10); Determine that a trajectory of the object (30) and a trajectory of the vehicle (10) will intersect; and Displays (370, 380), via head-up displays (204), a graphical warning message (202) that alerts the driver (102) of the vehicle (10) to the object (30) that is outside the driver's (102) line of sight (104); the head-up displays (204) include an augmented reality head-up display (206) and a blackout head-up display (208); the display (370, 380), via the head-up displays (204), includes a graphical warning message (202) that alerts the driver (102) of the vehicle (10) to the object (30) that is outside the driver's (102) line of sight (104): Creating an augmented reality image overlay (210); Creating a virtual image (212a, 212b); and Simultaneous projection of the augmented reality image overlay (210) and the virtual image (212a, 212b) onto a windshield (18) of the vehicle (10); characterized by , that the projection of the augmented reality image overlay (210) and the virtual image (212a, 212b) onto the windshield (18) of the vehicle (10) includes the projection of the augmented reality image overlay (210) onto a transparent part (24) of the windshield (18) and the projection of the virtual image (212a, 212b) onto a blackout part (26) of the windshield (18). [2] Method according to claim 1, wherein the augmented reality image overlay (210) differs from the virtual image (212a, 212b). [3] Method according to claim 1, wherein the sensor system (16) comprises one or more of the following elements: Cameras; a radio-based detection and distance measurement system (RADAR); and a light-based detection and distance measurement system (LIDAR). [4] Method according to claim 1, wherein determining that a trajectory of the object (30) and a trajectory of the vehicle (10) will intersect comprises: Receiving (310) vehicle data (22); Calculate (330), based on the sensor data (20), the trajectory of the object (30); Calculate (340), based on the vehicle data (22), the vehicle's trajectory (10); and Determine (350) whether the trajectory of the object (30) and the trajectory of the vehicle (10) intersect. [5] Method according to claim 4, wherein the operations further comprise determining (360), on the basis of the trajectory of the object (30) and the trajectory of the vehicle (10), a time until a collision between the object (30) and the vehicle (10). [6] Method according to claim 5, wherein the operations further comprise generating, on the basis of the time until the collision between the object (30) and the vehicle (10), the graphical warning message (202). [7] Method according to claim 1, wherein the display (370, 380), via the head-up displays (204), of the graphic warning message (202) which alerts the driver (102) of the vehicle (10) to the object (30) which is outside the line of sight (104) of the driver (102) comprises displaying the graphic warning message (202) on a windshield (18) of the vehicle (10) to indicate a direction of the object (30).

Images