System and method for notifying vehicle occupants of the severity and location of potential vehicle threats

Abstract

An exemplary method includes detecting a potential threat to the vehicle external to the vehicle, determining a severity level of the potential threat and a location of the potential threat relative to the vehicle, and displaying an indication of the potential threat on an electronic display in a specific portion of the electronic display corresponding to the location, the specific portion being one of a plurality of different portions of the electronic display each corresponding to a different threat location, and the specific display attribute being one of a plurality of different display attributes corresponding to a different severity level. Corresponding systems for detecting and displaying notification of a potential threat are also disclosed.

Description

[Technical Field]

[0001] This application relates to vehicles, and more particularly to a system and method for notifying vehicle occupants of the severity and location of potential threats to the vehicle.

[0002] (CROSS-REFERENCE TO RELATED APPLICATIONS) This disclosure claims the benefit of U.S. Provisional Application No. 63 / 058,208, filed July 29, 2020, the disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0003] Drivers of vehicles, such as commercial vehicles, are often presented with large amounts of information describing the environment surrounding the vehicle. This information may be presented on various displays within the vehicle, such as the vehicle instrument cluster and / or the displays of a camera monitor system (CMS) that replaces or supplements traditional vehicle mirrors. For example, a commercial vehicle may include multiple sensors, such as radar, cameras, ultrasonic, and other sensors, to understand the environment around the vehicle. The information provided to the driver may come from many sources, such as road data received wirelessly and / or the above sensors. A challenge is to extract this information so that it can be presented to the driver in a manner that minimizes driver distraction. Summary of the Invention

[0004] A method according to an example of the present disclosure includes detecting a potential threat to a vehicle external to the vehicle and determining a severity level of the potential threat and a location of the potential threat relative to the vehicle. The method includes displaying an indication of the potential threat on an electronic display at a specific portion of the electronic display corresponding to the location, the specific portion being one of a plurality of different portions of the electronic display each corresponding to a different threat location, and the specific display attribute being one of a plurality of different display attributes corresponding to a different severity level.

[0005] In a further embodiment of any of the above embodiments, determining a location of the potential threat relative to the vehicle includes determining which of a plurality of boundary areas of the vehicle is closest to the potential threat, and a particular portion of the electronic display corresponds to the determined boundary area.

[0006] In a further embodiment of any of the above embodiments, each of the plurality of display portions is a boundary region of the electronic display, each corresponding to a respective one of the boundary areas of the vehicle.

[0007] In a further embodiment of any of the above embodiments, the plurality of bounding regions include an upper left bounding region of the electronic display corresponding to a front left corner area of ​​the vehicle, an upper right bounding region of the electronic display corresponding to a front right corner area of ​​the vehicle, a lower left bounding region of the electronic display corresponding to a rear left area of ​​the vehicle, and a lower right bounding region of the electronic display corresponding to a rear right area of ​​the vehicle.

[0008] In a further embodiment of any of the above embodiments, the plurality of display attributes comprises a plurality of different colors, each corresponding to a different severity level.

[0009] In a further embodiment of any of the above embodiments, the plurality of colors includes a first color corresponding to a first severity level and a second color corresponding to a second severity level different from the first color and lower than the first severity level, and the method includes using a third color different from the first color and the second color to illuminate a boundary region of the electronic display corresponding to a boundary area of ​​the vehicle where no potential threats are detected.

[0010] In a further embodiment of any of the above embodiments, the method includes displaying a schematic image of the vehicle and a schematic image of the potential threat on the electronic display.

[0011] In a further embodiment of any of the above embodiments, the electronic display includes one or more of an instrument cluster display and a camera mirror replacement display.

[0012] In a further embodiment of any of the above embodiments, the plurality of display attributes includes a plurality of display patterns or sequences, each corresponding to a different severity level.

[0013] According to one example of the present disclosure, a vehicle system includes an electronic display configured to be viewed by a vehicle driver, at least one sensor configured to acquire information indicative of an environment surrounding the vehicle, and a computing device operably connected to the electronic display and the at least one sensor. The computing device is configured to determine a potential threat to the vehicle, a severity level of the potential threat, and a location of the potential threat relative to the vehicle based on the information from the at least one sensor. The computing device is configured to display an indication of the potential threat on the electronic display in a specific portion of the electronic display corresponding to the location, the specific portion having a specific display attribute corresponding to the severity level. The specific portion is one of a plurality of different portions of the electronic display, each corresponding to a different threat location, and the specific display attribute is one of a plurality of different display attributes corresponding to a different severity level.

[0014] In a further embodiment of any of the above embodiments, to determine the location of the potential threat, the computing device is configured to determine which of a plurality of boundary areas of the vehicle is closest to the potential threat, and a particular portion of the electronic display corresponds to the determined boundary area.

[0015] In a further embodiment of any of the above embodiments, each of the plurality of display portions is a boundary region of the electronic display, each corresponding to a respective one of the boundary areas of the vehicle.

[0016] In a further embodiment of any of the above embodiments, the plurality of bounding regions include an upper left bounding region of the electronic display corresponding to a front left corner area of ​​the vehicle, an upper right bounding region of the electronic display corresponding to a front right corner area of ​​the vehicle, a lower left bounding region of the electronic display corresponding to a rear left area of ​​the vehicle, and a lower right bounding region of the electronic display corresponding to a rear right area of ​​the vehicle.

[0017] In a further embodiment of any of the above embodiments, the plurality of display attributes comprises a plurality of different colors, each corresponding to a different severity level.

[0018] In a further embodiment of any of the above embodiments, the plurality of colors includes a first color corresponding to a first severity level and a second color corresponding to a second severity level different from the first color and lower than the first severity level, and the computing device is configured to use a third color different from the first color and the second color to illuminate a boundary region of the electronic display corresponding to a boundary area of ​​the vehicle where no potential threats are detected.

[0019] In a further embodiment of any of the above embodiments, the computing device is configured to display a schematic image of the vehicle and a schematic image of the potential threat on the electronic display.

[0020] In a further embodiment of any of the above embodiments, the electronic display includes one or more of an instrument cluster display and a camera mirror replacement display.

[0021] In a further embodiment of any of the above embodiments, the plurality of display attributes includes a plurality of display patterns or sequences, each corresponding to a different severity level.

[0022] A method according to an example of the present disclosure includes steps of detecting a potential threat to the vehicle that is external to the vehicle, determining a severity level of the potential threat and a location of the potential threat relative to the vehicle, selecting one of a plurality of boundary areas of an electronic display within the vehicle based on the location, and illuminating the selected boundary area of ​​the electronic display using a color corresponding to the severity level of the potential threat.

[0023] In a further embodiment of any of the above embodiments, each of the plurality of boundary regions corresponds to each of a plurality of boundary areas of the vehicle.

[0024] The embodiments, examples and alternatives of the preceding paragraphs, the claims, or the following description and drawings, including any of their various aspects or their respective individual features, may be taken independently or in any combination. Features described in connection with one embodiment are applicable to all embodiments, except where such features are incompatible. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a schematic diagram of a system for detecting and providing notification of potential threats to a vehicle.

[0026] [Figure 2] FIG. 1 is a schematic diagram of an exemplary electronic display location within a vehicle cab.

[0027] [Figure 3] FIG. 1 is a schematic diagram of exemplary sensor locations on a vehicle.

[0028] [Figure 4]1 is a flowchart of an exemplary method for notifying vehicle occupants of the severity and location of a potential threat to the vehicle.

[0029] [Figure 5] 1 is a first example image illustrating the severity of a potential threat to a vehicle and the location of a potential threat to a vehicle.

[0030] [Figure 6] 10 is a second example image showing the severity of a potential threat to a vehicle and the location of a potential threat to a vehicle. DETAILED DESCRIPTION OF THE INVENTION

[0031] 1 is a schematic diagram of a system 10 for detecting potential threats to a vehicle 12 and providing notification of the potential threat indicating the size of the threat and the direction of the threat relative to the vehicle. The system 10 includes a computing device 14 (e.g., an electronic control unit "ECU") in communication with a plurality of displays 18A-E and a plurality of sensors 16A-N observable by a driver within the cab of the vehicle 12.

[0032] The sensors 16A-N may include, for example, cameras, radar sensors, and / or ultrasonic sensors (e.g., lidar). In the example of Figure 1, the displays 18 include two camera monitor system (CMS) displays 18A-B that are part of a CMS (e.g., Stoneridge®'s MirrorEye®) that complement or replace traditional exterior mirrors, an instrument cluster display 18C, an infotainment display 18D, and a rearview display 18E located in the traditional location of the rearview mirror. Of course, it will be understood that other types and quantities of electronic displays 18 may be used.

[0033] In one example, multiple sensors 16A-N communicate with computing device 14 via a communication bus 20, such as a controller area network (CAN) bus.

[0034] Computing device 14 includes a processor 22 operably connected to memory 24, a communications interface 26, and a wireless transceiver 29. Processor 22 includes processing circuitry to process data from sensors 16, detect potential threats to vehicle 12, determine a severity level of the potential threats, and determine a location of the potential threats relative to vehicle 12. Processor 22 may include, for example, one or more microprocessors, microcontrollers, application specific integrated circuits (ASICs), etc.

[0035] The memory 24 may include any one or combination of volatile memory elements (e.g., random access memory (RAM, such as DRAM, SRAM, SDRAM, VRAM, etc.)) and / or non-volatile memory elements (e.g., ROM, hard drive, tape, CD-ROM, etc.). Additionally, the memory 24 may incorporate electronic, magnetic, optical, and / or other types of storage media. The memory 24 may have a distributed architecture in which various components are located remotely from one another but are accessible by the processor 22. The memory 24 includes a neural network 28 for processing data from the sensors 16 and detecting potential threats to the vehicle 12, for example, using object detection techniques known in the art.

[0036] The communication interface 26 is configured to facilitate communication between the computing device 14 , the sensor 16 , and the display 18 .

[0037] Wireless transceiver 29 is configured to receive data regarding information about road obstacles, road conditions, nearby accidents, GPS information, etc. In one example, computing device 14 is operable to detect potential threats based on information received from wireless transceiver 29 (e.g., if vehicle 12 is approaching a road obstacle based on the obstacle and GPS data of vehicle 12).

[0038] FIG. 2 is a schematic diagram of exemplary locations of displays 18A-E within a cab 30 of an example vehicle 12. In the example of FIG. 2, the cab 30 is within the tractor portion of a commercial vehicle. As shown in FIG. 2, CMS displays 18A-B are mounted on A-pillars 32A-B of the vehicle 12. An instrument cluster display 18C is located in front of the driver's seat 34. An infotainment display 18D is located on the dashboard 36 of the vehicle cab 30. A rearview display 18E is located in the position of a traditional rearview mirror. Of course, these are merely exemplary displays and display locations, and it will be understood that other quantities of displays and / or other display locations within the vehicle cab 30 may be used.

[0039] FIG. 3 is a schematic diagram of exemplary locations of sensors 16 and the proximity of their respective detection areas 40 for an example commercial vehicle 12 including a tractor 41 and trailer 42. As shown in FIG. 3, the vehicle 12 has a front side 17A, a rear side 17B, a left side 17C, and a right side 17D. As used herein, relative positions such as left, right, front, and rear are described with reference to a vehicle operator facing forward of the vehicle 12. The rear corners of the trailer 42 of the vehicle 12 are provided with sensors 16A-B having associated detection areas 40A-B. The front corners of the tractor 41 of the vehicle 12 are provided with sensors 16C-D having associated detection areas 40C-D. The rear of the trailer 42 of the vehicle 18 is provided with sensor 16E having associated detection area 40E. The front of the tractor 41 of the vehicle 12 is provided with sensor 16F having associated detection area 40F. Sensors 16G-H are CMS camera sensors and have associated detection areas 40G-H.

[0040] The vehicle 12 includes multiple boundary areas 44A-D, including a front left boundary area 44A, a front right boundary area 44B, a rear left boundary area 44C, and a rear right boundary area 44D. These exemplary boundary areas 44 are described in more detail below. While several exemplary boundary areas are shown in FIG. 3, it is understood that these are non-limiting examples and that other boundary areas may be used.

[0041] 4 is a flowchart 100 of an exemplary method performed by computing device 14 to notify vehicle occupants of the severity and location of a potential threat to vehicle 12. Computing device 14 receives information about the environment surrounding vehicle 12 from multiple sensors 16A-N and / or wireless transceiver 29 (step 102) and analyzes the information to determine whether a potential threat has been detected (step 104).

[0042] Step 104 may include performing object detection and comparing the data received from sensors 16 with predefined threat criteria to determine whether a collision is possible, etc. For example, if another vehicle is within X feet of vehicle 12 and approaching vehicle 12 at a speed of Y, it may constitute a severity 2 threat, while if the same vehicle is approaching vehicle 12 at a speed of Z, which is slower than speed Y, it may constitute a severity 1 threat or no threat due to the low likelihood of a collision. Other potential vehicle threats may include road obstacles that may be detected from the data received by wireless transceiver 29. Some example road obstacles may include stranded or crashed vehicles and / or adverse road conditions, such as known ice patches and / or flooding.

[0043] If no potential threat is detected (step 104: No), computing device 14 continues to receive and monitor the information of step 102 to monitor for potential threats.

[0044] Alternatively, if a potential threat is detected (“Yes” at step 104), computing device 14 determines the severity level of the potential threat and also determines the location of the potential threat relative to vehicle 12 (step 106).

[0045] Computing device 14 determines a portion of one or more electronic displays 18 that corresponds to the determined location of the potential threat (step 108). The particular portion determined in step 108 for each of one or more electronic displays 18 is one of a plurality of different portions of electronic display 18 that correspond to different threat locations.

[0046] The computing device 14 also determines a display attribute corresponding to the severity level of the potential threat (step 110). The particular display attribute determined in step 110 is one of a number of different display attributes (e.g., color, pattern, and / or sequence of blinking, etc.) corresponding to different severity levels. In the blinking example, the rate of blinking and / or the color used for blinking may vary depending on the severity level.

[0047] Computing device 14 displays an indication of the potential threat on electronic display 18 using the determined portion of electronic display 18 in step 108 and the determined display attributes in step 110 (step 112).

[0048] In one example, determining the location of the potential threat relative to the vehicle in step 106 includes determining which of multiple boundary areas 44A-D of the vehicle 12 is closest to the potential threat. In this regard, it will be understood that the boundary areas 44A-D shown in FIG. 3 are merely examples, and that other boundary areas may also be used (e.g., between adjacent ones of areas 44A-D).

[0049] In one example, the computing device 14 provides an audible and / or physical warning (eg, a vibration in the steering wheel) in addition to providing a notification on the electronic display 18.

[0050] In one example, the particular portion of the electronic display determined in step 108 corresponds to border area 44 of electronic display 18. One such example is shown in FIG.

[0051] 5, an exemplary image 48 of electronic display 18 is shown. Image 48 includes multiple boundary regions 50A-D. In the example of FIG. 5, display boundary region 50A corresponds to vehicle front-left boundary area 44A, display boundary region 50B corresponds to vehicle front-right boundary area 44B, display boundary region 50C corresponds to vehicle rear-left boundary area 44C, and display boundary region 50D corresponds to vehicle rear-right boundary area 44D.

[0052] Image 18 is an overhead view of vehicle 12, illustrated schematically as a truck 52, and illustrates a number of objects 54, 56, and 58 representing objects detected in the vicinity of vehicle 12. The objects include a first vehicle 54, a second vehicle 56, and a motorcycle 58.

[0053] In the following description, it is assumed that vehicles 54 and 56 are determined to be potential threats, with vehicle 54 having a severity of 2 (e.g., because it is within a predetermined distance of truck 52 and is approaching the truck at a speed above a predetermined threshold), vehicle 56 having a severity of 1, which is lower than severity 2 (e.g., because it is within a predetermined distance of truck 52 and is accelerating toward truck 52), and motorcycle 58 not being considered a potential threat (e.g., having a threat level below a predefined threshold).

[0054] Computing device 14 determines that vehicle 54 is closest to vehicle boundary area 44D, and thereby selects display boundary region 50D corresponding to vehicle boundary area 44D, and displays display boundary region 50D with a display attribute corresponding to severity 2. In the example of FIG. 5, this is cross-hatching, but may alternatively or additionally include a color such as red.

[0055] Computing device 14 determines that vehicle 56 is closest to vehicle boundary area 44C, thereby selecting display boundary region 50C corresponding to vehicle boundary area 44C, and illustrating display boundary region 50C with display attributes corresponding to severity level 1. In the example of FIG. 5, this is non-cross-hatched, but may alternatively or additionally include a color such as yellow.

[0056] Computing device 14 determines that motorcycle 58 is not a potential threat to vehicle 12 (e.g., because motorcycle 58 is moving away from truck 52) and therefore displays motorcycle 58 with a display attribute indicative of a non-threat. In the example of Figure 5, this is a dotted shading, but may alternatively or additionally include a color such as green. Boundary area 50B also does not have a corresponding potential threat, and therefore is displayed with the same display attribute as displayed boundary region 50A.

[0057] FIG. 6 illustrates an example of how method 100 may be implemented in conjunction with a CMS display 18B, such as that shown in FIG. 2, without utilizing a bird's-eye view. Display 18B includes a first display area 60A corresponding to a Class II view and a second display area 60B corresponding to a Class IV wide-angle view. Each display area 60A-B depicts a truck 62. In the following discussion, it is assumed that computing device 14 determines that truck 62 corresponds to a potential threat of severity 1 and is closest to rear-right vehicle boundary area 44D. Computing device 14 renders display boundary region 50D corresponding to rear-right vehicle boundary area 44D using display attributes corresponding to severity 1. Optionally, display boundary regions 50A and 50C-D may be depicted using non-threat display attributes.

[0058] Prior art systems rely on notifications such as simple lights (e.g., blind spot detection systems that activate corner lights in the rearview mirror) or audio warnings (e.g., beeps or tones that vary in volume based on the severity of the perceived threat). Still other systems vibrate the seat or steering wheel. However, these warnings are insufficient on their own for vehicles such as commercial vehicles, which may include one or more trailers, because the warnings do not convey the location of the threat or the severity of the danger sufficiently to provide the driver with time to act, and the size of the vehicle can render these warnings ineffective.

[0059] Furthermore, in commercial vehicles, where the amount of area that needs to be observed by the vehicle's sensors is greater than in passenger cars, a lack of specificity in the warnings can result in the system being disabled by the driver. Additionally, some drivers may find non-specific warnings distracting.

[0060] The present disclosure improves upon prior art systems by clearly communicating information regarding the severity and location of potential threats to the vehicle, allowing the driver to assess the location and severity of the threat and respond quickly (e.g., in as little as 0.5 seconds).

[0061] While exemplary embodiments have been disclosed, a worker of ordinary skill in this art would recognize that certain modifications would come within the scope of the present disclosure. For that reason, the following claims should be studied to determine the scope and content of the present disclosure.

Claims

1. A step of displaying a first view and a second view having a wider angle than the first view on an electronic display of a camera monitor system of a vehicle, the electronic display including a plurality of boundary regions, each of the plurality of boundary regions corresponding to a respective one of a plurality of boundary areas of the vehicle; detecting a potential threat to the vehicle external to the vehicle, the step including determining the presence of the potential threat by comparing data relating to the detected object with predefined threat criteria; determining a severity level of the potential threat and a location of the potential threat relative to the vehicle; if the potential threat is present, displaying on the electronic display an indication of the potential threat in one of the bounded areas of the electronic display corresponding to the location, the indication having particular display attributes corresponding to the severity level; Including, the particular display attribute is one of a plurality of different display attributes corresponding to different severity levels; Determining the location of the potential threat relative to the vehicle includes determining which of a plurality of boundary areas of the vehicle is closest to the potential threat; The method wherein the particular portion of the electronic display corresponds to a determined boundary area.

2. The plurality of boundary regions may include: an upper left boundary region of the electronic display corresponding to the front left corner area of ​​the vehicle; an upper right boundary region of the electronic display corresponding to the front right corner area of ​​the vehicle; a lower left boundary region of the electronic display corresponding to the rear left area of ​​the vehicle; and a lower right boundary area of ​​the electronic display corresponding to the rear right area of ​​the vehicle; The method of claim 1 , comprising:

3. The method of claim 1 , wherein the plurality of display attributes comprises a plurality of different colors, each corresponding to a different severity level.

4. the plurality of colors includes a first color corresponding to a first severity level and a second color corresponding to a second severity level different from the first color and lower than the first severity level; 4. The method of claim 3, further comprising using a third color different from the first color and the second color to illuminate a boundary region of the electronic display corresponding to a boundary area of ​​the vehicle where no potential threats are detected.

5. 10. The method of claim 1, further comprising displaying on the electronic display a schematic image of the vehicle and a schematic image of the potential threat, both separate from an indication of the potential threat on the electronic display.

6. The method of claim 1 , wherein the electronic display comprises one or more of an instrument cluster display and a mirror replacement video display.

7. The method of claim 1 , wherein the plurality of display attributes comprises a plurality of display patterns or sequences, each corresponding to a different severity level.

8. The displaying step includes: displaying a first indication regarding a first potential threat on a first portion of the plurality of portions of the electronic display corresponding to a first side of the vehicle; and displaying a second indication regarding a second potential threat on a second portion of the plurality of portions of the electronic display corresponding to a second side of the vehicle. The method of claim 1 , comprising:

9. An electronic display of a camera monitor system, the electronic display being observed by a driver of a vehicle and configured to display a first view and a second view that is wider than the first view on the electronic display, the electronic display including a plurality of boundary regions, each of the plurality of boundary regions corresponding to a respective one of a plurality of boundary areas of the vehicle; at least one sensor configured to acquire information indicative of an environment surrounding the vehicle; a computing device operably connected to the electronic display and the at least one sensor; wherein the computing device comprises: determining a potential threat to the vehicle, a severity level of the potential threat, and a location of the potential threat relative to the vehicle based on the information from the at least one sensor, wherein the presence of the potential threat is determined by comparing the information to predefined threat criteria; and If the potential threat is present, displaying on the electronic display an indication of the potential threat with particular display attributes corresponding to the severity level in one of the bounded areas of the electronic display corresponding to the location. configured to: the particular display attribute is one of a plurality of different display attributes corresponding to different severity levels; To determine the location of the potential threat, the computing device is configured to determine which of a plurality of boundary areas of the vehicle is closest to the potential threat; The particular portion of the electronic display corresponds to a determined boundary area.

10. The plurality of boundary regions may include: an upper left boundary region of the electronic display corresponding to the front left corner area of ​​the vehicle; an upper right boundary region of the electronic display corresponding to the front right corner area of ​​the vehicle; a lower left boundary region of the electronic display corresponding to the rear left area of ​​the vehicle; and a lower right boundary area of ​​the electronic display corresponding to the rear right area of ​​the vehicle; The system of claim 9 , comprising:

11. The system of claim 9 , wherein the plurality of display attributes comprises a plurality of different colors, each corresponding to a different severity level.

12. 12. The system of claim 11, wherein the plurality of colors includes a first color corresponding to a first severity level and a second color corresponding to a second severity level different from the first color and lower than the first severity level, and the computing device is configured to use a third color different from the first color and the second color to illuminate a boundary region of the electronic display corresponding to a boundary area of ​​the vehicle where no potential threats are detected.

13. 10. The system of claim 9, wherein the computing device is configured to display on the electronic display a schematic image of the vehicle and a schematic image of the potential threat, both of which are separate from an indication of the potential threat on the electronic display.

14. The system of claim 9 , wherein the electronic display comprises one or more of an instrument cluster display and a mirror replacement video display.

15. The system of claim 9 , wherein the plurality of display attributes comprises a plurality of display patterns or sequences, each corresponding to a different severity level.

16. the computing device, displaying a first indication regarding a first potential threat on a first portion of the plurality of portions of the electronic display corresponding to a first side of the vehicle; and displaying a second indication regarding a second potential threat on a second portion of the plurality of portions of the electronic display corresponding to a second side of the vehicle. The system of claim 9 configured to:

17. A step of displaying a first view and a second view having a wider angle than the first view on an electronic display of a camera monitor system of a vehicle, the electronic display including a plurality of boundary regions, each of the plurality of boundary regions corresponding to a respective one of a plurality of boundary areas of the vehicle; detecting a potential threat to the vehicle external to the vehicle, the step including determining the presence of the potential threat by comparing data relating to the detected object with predefined threat criteria; determining a severity level of the potential threat and a location of the potential threat relative to the vehicle; selecting one of the plurality of boundary areas on an electronic display within the vehicle corresponding to the location, each of the plurality of boundary areas corresponding to a different potential threat surrounding the vehicle; illuminating a selected border area of ​​the electronic display when the potential threat is present using a color corresponding to the severity level of the potential threat based on the detecting, determining, and selecting steps; and A method comprising:

Images